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FOREWORD 


One  long-range  research  goal  of  the  U.S.  Army  Research  Institute,  and 
subsequently  the  Human  Research  and  Engineering  Directorate  of  the  U.S.  Army 
Research  Laboratory,  is  to  develop  methods  for  estimating  the  resources 
required  to  develop  and  implement  the  training  system  to  support  a  new 
materiel  system.  The  objective  is  to  influence  the  design  of  that  materiel 
system  early  in  its  development  program  by  showing  the  impact  of  alternate 
materiel  system  designs  on  training  development  and  implementation  costs.  As 
an  initial  step  in  the  development  of  such  methods,  a  broad-based  review  of 
the  literature  was  conducted  to  establish  the  state  of  the  art  and  to  identify 
what  models  and  methods  currently  exist  that  might  provide  useful  inputs  to 
the  program.  Given  the  breadth  of  the  search,  this  report  has  been  assembled 
in  the  hope  that  others  might  find  the  abstracts  under  one  or  more  topics 
germane  to  their  concerns. 

The  reviewed  literature  is  organized  under  the  topic  area  headings  of 
training  system  models;  training  requirements  determination,  and  design  and 
development  methods;  prediction  and  evaluation  of  student  and  media 
performance  effectiveness;  training  media;  factors  influencing  outcomes; 
costing  and  resourcing;  and  miscellaneous.  An  analysis  of  this  literature  is 
presented  in  Volume  I  of  this  report. 

This  research  has  been  performed  under  the  auspices  of  Research  Task 
1209,  Soldier-Equipment  Considerations  in  Military  Occupational  Specialty 
(MOS)  Design.  Work  under  that  task  is  focusing  on  the  development  of  methods 
to  optimize  the  clusters  of  tasks  (i.e.,  task  structure,  assigned  to  MOSs  when 
changes  occur  in  doctrine  or  as  a  function  of  force  modernization) .  One  of 
the  major  impacts  of  changes  in  MOS  task  structure,  and  one  that  must  be 
accounted  for,  is  on  the  design  of  the  training  system  supporting  the  MOSs. 

The  research  of  MOS  restructuring  and  this  review  of  the  literature  will 
provide  inputs  to  the  long-range  goal  of  developing  training  system  resource 
estimation  methods  that  will  affect  materiel  system  design  choices. 
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EXECUTIVE  SUMMARY 


RESEARCH  REQUIREMENTS 

The  design  of  a  prime  materiel  system  from  the  standpoint  of  its 
operability  and  maintainability  on  the  part  of  the  soldier  largely  determines 
the  manpower,  personnel,  and  training  (MRT)  resource  requirements  needed  to 
support  the  system.  It  is  one  of  the  tenets  of  the  manpower  and  personnel 
integration  (MANPRINT)  program  that,  to  the  extent  that  the  MPT  resource 
requirements  and  reasonableness  thereof  can  be  estimated  early  in  the  prime 
system's  development  program,  the  greater  the  likelihood  that  prime  system 
designs  that  are  costly  to  support  and  noneffective  will  be  avoided.  That  is, 
alternate  system  designs  will  be  considered  that  are  less  MPT  resource 
intensive . 

A  long-range  goal  of  the  U.S.  Army  Research  Institute,  and  subsequently 
the  Human  Research  and  Engineering  Directorate  of  the  U.S.  Army  Research 
Laboratory,  is  to  develop  methods  for  estimating  the  resources  required  to 
develop  and  implement  the  training  system  to  support  a  new  materiel  system 
design  early  in  that  system's  development.  A  necessary  first  step  was  to 
review  the  literature  to  determine  the  state  of  the  art  and  to  identify  what 
models  and  methods  exist  that  might  provide  useful  inputs  to  the  program. 


PROCEDURE 

The  definition  of  "estimation”  used  in  the  search  for  documents  was  a 
broad-based  one.  An  attempt  was  made  to  uncover  any  ideas  that  bore  directly 
on  the  prediction,  estimation,  or  evaluation  of  the  design,  cost,  performance, 
or  effectiveness  of  a  training  system.  Likewise,  there  was  no  attempt  to 
restrict  the  search  to  training  "systems”  per  se.  While  we  feel  that  the 
training  "system”  that  supports  a  materiel  end  item  may  encompass  many 
subsystems  and  elements,  many  authors  do  not  use  the  term  in  such  an  all- 
encompassing  manner.  Many  view,  for  example,  a  single  training  device, 
intelligent  tutoring  system,  or  other  medium  as  a  training  system,  even  though 
these  media  may  play  only  a  small  part  in  the  overall  training  programs, 
necessary  facilities,  and  so  forth  thst  support  a  materiel  system. 

The  primary  sources  reviewed  in  the  literature  search  include  the 
proceedings  of  the  annual  Human  Factors  Society  conferences.  Psychological 
Abstracts,  and  the  authors'  personal  libraries.  In  addition,  some  documents 
were  obtained  from  the  National  Technical  Information  Center  after  a  search 
was  performed  under  such  terms  as  training  systems  and  training  resource 
estimation. 


FINDINGS 

One  hundred  seventy-five  reports  were  reviewed.  The  abstracts  fell  into 
several  topic  areas  and  were  organized  accordingly.  The  major  topic  areas 
include  training  system  models;  training  requirements  determination,  and 
design  and  development  methods;  prediction  and  evaluation  of  student  and  media 
performance  effectiveness;  training  media;  factors  influencing  outcomes; 
costing  and  resourcing;  and  miscellaneous.  Several  documents  fell  into  more 
than  one  category.  When  this  occurred,  the  abstracts  have  been  presented 
under  the  first  topic  heading  and  then  cross  referenced  under  the  subsequent 
topic  headings. 
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USE  OF  FINDINGS 

This  annotated  bibliography  has  been  assembled  in  the  hope  that  others 
might  find  the  abstracts  under  one  or  more  topics  germane  to  and  useful  for 
addressing  their  concerns.  The  documents  were  analyzed  and  formed  the  basis 
for  Volume  I.  Therein,  the  state  of  the  art  with  regard  to  such  models  is 
described,  and  research  issues  are  identified. 
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APPLYING  TRAINING  SYSTEM  ESTIMATION  MODELS  TO  ARMY  TRAINING: 
VOLUME  II.  AN  ANNOTATED  BIBLIOGRAPHY  1970-1990 


INTRODUCTION 

Purpose 

One  long-range  research  goal  of  the  U.S.  Amy  Research  Institute,  and 
subsequently  the  Human  Research  and  Engineering  Directorate  of  the  U.S.  Amy 
Research  Laboratory,  is  to  develop  methods  for  estimating  the  resources 
required  to  develop  and  implement  the  training  system  required  to  support  a 
new  materiel  system.  The  objective  is  to  influence  the  design  of  that 
materiel  system  early  in  its  development  program  by  showing  the  impact  of 
alternate  materiel  system  designs  on  training  development  and  implementation 
costs.  As  an  initial  step  in  the  development  of  such  methods,  a  review  of  the 
literature  was  conducted  to  establish  the  state  of  the  art  and  to  identify 
what  models  and  methods  currently  exist  that  might  provide  useful  inputs  to 
the  program.  This  report  presents  abstracts  of  the  reviewed  literature.  The 
analysis  of  this  literature  is  presented  in  Volume  I  of  this  report. 

Even  though  the  ultimate  goal  of  this  program  is  focused  on  methods  for 
resource  estimation,  the  definition  of  training  system  "estimation"  used  in 
this  literature  search  was  purposely  a  much  broader  one.  An  attempt  was  made 
to  uncover  any  ideas  that  bore  directly  on  the  estimation  or  evaluation  of  the 
design,  cost,  performance,  or  effectiveness  of  a  training  system.  Likewise, 
there  was  no  attempt  to  restrict  the  search  to  training  "systems"  per  se. 

While  we  feel  that  the  training  "system"  that  supports  a  materiel  end  item  may 
encompass  many  subsystems  and  elements,  many  authors  do  not  use  the  term  in 
such  an  all-encompassing  manner.  Many  view,  for  example,  a  training  device, 
intelligent  tutoring  system,  or  other  medium  as  a  training  system,  even  though 
these  media  may  play  only  a  small  part  in  the  overall  training  programs, 
necessary  facilities,  and  so  forth  that  support  a  materiel  system. 

Given  the  breadth  of  the  search,  this  report  has  been  assembled  in  the 
hope  that  others  might  find  the  abstracts  under  one  or  more  topics  germane  to 
their  concerns.  The  primary  sources  reviewed  in  the  literature  search  include 
the  proceedings  of  the  annual  Human  Factors  Society  conferences.  Psychological 
Abstracts,  and  the  authors'  personal  libraries.  In  addition,  some  documents 
were  obtained  from  the  National  Technical  Information  Center  after  a  search 
was  performed  under  such  terms  as  training  systems  and  training  resource 
estimation. 


Organization  of  the  Report 

The  abstracts  are  organized  into  major  topic  areas  with,  in  some  cases, 
subheadings.  The  topics  include 

•  Training  system  models:  concepts  and  simulations  of  what  training 
systems  are  comprised  of  and  how  they  operate; 

•  Training  requirements  determination,  and  design  and  development 
methods:  materials  that  bear  on  the  development  of  training  programs  and 
systems,  including  general  analysis  methods;  alternate  strategies;  and  methods 
specific  to  crew  and  team  training  and  to  leader  and  instructor  training; 
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•  Prediction  and  evaluation  of  student  and  madia  performance 
effectiveness:  net bods  sad  findings  that  relate  to  the  (1)  a  priori 
prediction  of  efeetfeer  group*  of  students  and  specific  media  will  be  successful 
in  training  and  (21  evaluation  of  whether  they  were  successful  in  training; 

•  Training  mr  ‘la:  considerations  and  demonstrations  regarding  what 
features  night  be  at  -atagooua  to  include  in  a  training  medium;  methods  for 
selecting  aneng  media  choices;  and  information  specific  to  training  devices 
and  to  counter  "baaed  training  and  artificial  intelligence; 

•  factors  influencing  outcoams:  research  concerning  various  factors 
affecting  training  outcomes  that  did  not  fall  into  the  previous  categories 
(o.g.,  instructor  work  load,  individual  differences,  cooperative  learning); 

•  Costing  and  resourcing:  descriptions  of  models  of,  methods  for,  and 
considerations  regarding  the  determination  of  costs  to  develop  and  operate 
training  prograam  and  media;  and 

•  Miscellaneous:  amterials  that  were  related  to  the  general  issue  but 
did  not  fall  easily  into  any  of  the  previous  categories.  Several  reports 
attempt  to  integrate  subject  areas  related  to  training. 

Several  documents  fell  into  more  than  one  category.  When  this  was  the 
case,  the  abstract  appears  in  the  first  category  under  which  it  falls,  and 
reference  is  made  to  it  in  the  appropriate  subsequent  categories,  using  their 
item  numbers.  The  abstracts  are  not  presented  alphabetically  by  author  names. 
When  the  abstracts  are  reviewed,  date  was  the  most  meaningful  sequencing. 

Whole  areas  of  endeavor  and  individual  programs  have  flourished  and  evolved 
during  the  years.  The  first  three  topic  areas,  for  example,  have  evolved  in 
terms  of  more  and  more  detailed  procedures  in  general  with  tailor-made 
procedures  coming  into  being  for  some  specific  types  of  training  (e.g., 
development  methods  for  training  devices  and  embedded  training) .  In  the  area 
of  training  media,  the  evolution  into  increasingly  interactive  and  intelligent 
media  is  apparent .  It  is  only  by  presenting  the  abstracts  sequentially  by 
date  that  these  changes  in  the  state  of  the  art  become  apparent . 

For  those  readers  who  have  a  need  or  desire  to  search  for  a  particular 
author,  all  authors  are  listed  in  the  appendix.  They  are  listed 
alphabetically  and  referenced  to  the  text. 
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TRAINING  SYSTEM  MODELS 

096  Kaufman,  R.A.,  Corrigan,  R.E.,  &  Nunnelly,  C.L.  (1966) .  The  instruc¬ 
tional  system  approach  to  training.  Human  Factors.  £.(2),  157-162. 

A  generalized  model  for  an  Ir structional  System  is  offered,  as  well  as  a  model 
for  such  a  system  for  use  in  preparing  training  and  training  materials  for  the 
U.S.  Air  Force.  The  rationale  for  a  systematic  approach  to  training  is 
presented  along  with  a  discussion  of  a  relationship  between  an  Instructional 
System  and  Programmed  Instruction. 

015  Brock,  J.F.  (1978) .  Maintenance  training  and  simulation:  design 

processes  and  evaluation  criteria.  Proceedings  of  the  Human  Factors 
Soclety-22nd  Annual  Meeting.  260-266. 

There  is  general  agreement  among  instructional  designers  that  training  people 
to  maintain  complex  systems  has  unique  characteristics  not  found  in  other 
training  endeavors.  These  unique  characteristics  stem  from  the  kind  of 
cognitive  and  psychophysical  tasks  which  must  be  performed  by  the  system 
maintainer.  This  paper  discusses  the  uniqueness  required  of  maintenance 
training  systems  and  sets  out  a  model  for  designing  such  systems.  Examples 
from  basic  and  advanced  maintenance  training  systems  are  presented  with  a 
discussion  of  how  and  why  particular  instructional  strategies  were  selected. 
Alternative  strategy  selection  techniques  are  discussed. 

A  review  of  current  simulation  techniques  for  training  maintenance  tasks  is 
presented  in  the  context  of  general  design  considerations  for  maintenance 
training.  Computer-based  systems  are  compared  along  dimensions  of  cost, 
reliability,  modifiability,  hands-on  capability,  safety,  and  student 
performance  data. 

A  concurrent  problem  in  maintenance  training  is  the  determination  of  whether  a 
training  system  is,  in  fact,  functioning  efficiently  and  effectively. 
Efficiency  and  effectiveness  are  defined  and  a  model  for  measuring  each  is 
proposed.  A  section  of  the  paper  addresses  the  economic  issues  in  the 
evaluation  of  training. 

045  Finley,  D.L.,  Strasel,  H.C.,  Bloom,  R.L.,  6  Oates,  J.F.  (1982).  An 
analytic  process  method  for  defining  appropriate  measures  of  systems' 
performance  and  effectiveness.  Proceedings  of  the  Human  Factors  Scciety- 
2fith  Annual  Meeting,  835-839. 

A  definition  of  what  constitutes  a  training  system,  and  a  systematic  analytic 
process  for  better  defining  necessary  and  sufficient  measures  of  training  and 
other  systems  are  presented.  The  research  to  develop  the  definition  and 
process  was  accomplished  within  the  context  of  a  relatively  complex  Army  man- 
machine  system. 

114  Matlick,  R.K.,  Swezy,  R.W.,  &  Epstein,  K.I.  (1982).  Alternative  models 
for  individualized  armor  training:  I.  Interim  report:  Review  and 
analysis  of  the  literature.  Catalog  of  Selected  Documents  in  Psychology. 
12(4),  49. 
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163  Sticha,  P.J.,  Knerr  C.M.,  &  Goldberg,  S.L.  (1984).  Application  of 

simulation  and  modeling  to  army  training  management.  Proceedings  of  the 
Human  Factors  Society-  28th  Annual  Meeting.  1023-1027. 

Simulation  models  to  aid  the  training  manager  may  be  developed  at  three  levels 
of  increasing  generality:  task  performance  models,  task  training  models,  and 
training  system  models.  The  more  general  models  encompass  more  variables,  and 
hence,  are  more  useful  to  the  training  manager.  The  bulk  of  research  in  this 
area  has  concentrated  on  task  performance  models,  and  within  this  framework, 
has  studied  skill  retention.  The  findings  of  this  research  must  be  integrated 
with  system  cost  and  utility  information  to  produce  a  training  system  model. 
Development  of  such  a  model  can  provide  an  essential  set  of  tools  to  be 
incorporated  into  a  decision  support  system  for  training  management. 

004  Aldrich,  R.E.,  Strasel,  H.C.,  &  Ditzian,  J.L.  (1986).  The  Army  unit 
training  model  and  guidelines  for  the  integration  of  Embedded  Training 
(ET) .  Proceedings  of  the  Human  Factors  Society-  30th  Annual  Meeting. 
1019-1023. 

Embedded  Training  (ET)  is  a  new  training  alternative  for  individual  and 
collective  sustainment  training  in  Army  systems  at  the  unit  level.  This  paper 
outlines  the  existing  Army  Unit  Training  Model,  ET  integration  requirements, 
guidelines  for  the  integration  of  ET  into  the  existing  components  of  the  Army 
Unit  Training  Model  and  the  creation  of  a  new  ET  specific  component . 

148  Shapiro,  R.G.,  Bloom,  R.L.,  &  Oates,  J.F.  (1966).  The  computer-aided 
analytic  process  mode];  Operations  handbook  for  the  analytic  process 
model  demonstration  package  (ARI  Research  Note  86-06)  .  Alexandria,  VA: 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

The  computer-aided  APM  demonstration  provides  the  analyst  with  the  opportunity 
to  perform  a  thorough  analysis  of  a  system,  while  the  computer  keeps  track  of 
the  analysis,  and  insures  that  the  analyst  examines  the  parts  of  the  data  base 
which  are  of  interest.  This  is,  however,  a  demonstration  package  which  can 
only  process  small  data  bases.  Because  the  package  is  implemented  on  an  Apple 
II  Plus,  processing  is  relatively  slow.  An  explanation  of  the  APM  listings  of 
data  sets  derived  using  the  APM  and  recommendations  for  further  development  of 
the  APM  model  appear  in  the  companion  volume  -  "The  Analytic  Process  Model  for 
System  Design  and  Measurement:  A  Computer-Aided  Tool  for  Analyzing  Training 
Systems  and  Other  Human-Machine  Systems." 

149  Shapiro,  R.G.  (1986) .  The  computer-aided  analytic  process  model: 

Appendix  to  the  Operations  Handbook  for  the  APM  demonstration  package 
(ARI  Research  Note  86-07).  Alexandria,  VA:  U.S.  Army  Research  Institute 
f cr  the  Behavioral  and  Social  Sciences . 

The  computer-aided  APM  demonstration  package  provides  the  analyst  with  the 
opportunity  to  perform  a  thorough  analysis  of  a  system  while  the  computer 
keeps  track  of  the  analysis  and  insures  that  the  analyst  examines  the  parts  of 
the  data  base  which  are  of  interest.  This  is,  however,  a  demonstration 
package  which  can  only  process  small  data  bases.  Because  the  package  is 
implemented  on  an  Apple  II  Plus,  processing  is  relatively  slow.  An  explanation 
of  the  APM,  listings  of  the  data  sets  derived  using  the  APM  and  recommenda¬ 
tions  for  further  development  of  the  APM  appear  in  the  companion  volume-  "The 
Analytic  Process  Model  for  System  Design  and  Measurement:  A  Computer-Aided 
Tool  for  Analyzing  Training  Systems  and  Other  Human-Machine  Systems."  A 
separate  companion  volume-  "The  Computer-Aided  Analytic  Process  Model: 
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Operations  Handbook  for  the  APM  Demonstration  Package"  is  also  available  under 
separate  cover.  The  present  volume,  which  is  an  Appendix  to  the  Operations 
Handbook,  contains  the  actual  PASCAL  computer  code  listings.  Disks  containing 
this  code  and  the  data  bases  in  machine-readable  format  are  also  available. 

141  Roth,  J.T.  (1988).  Implementing  Embedded  Training  (ET)  : _ Volume  3  of  10: 

The  Role  of  ET  in  the  Training  System  Concept  (Research  Product  No.  88- 
13).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

This  product  provides  a  step-by-step  procedure  for  developing  a  training 
system  concept  very  early  in  the  materiel  acquisition  process.  The  training 
system  concept  produced  outlines  appropriate  roles  for  Embedded  Training  and 
Stand-alone  Training  Devices  in  supporting  hands-on  training  for  a  materiel 
system,  with  respect  to  nine  specific  training  needs  or  situations. 

082  Hinton,  W.M.,  Braby,  R.,  Feuge,  R.L.,  Stults,  A.H.,  Evans,  S.M.,  Gibson, 
M.R.,  &  Zaldo,  W.T.  (1990).  A  design  architecture  for  an  integrated 
training  system  decision  support  system.  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 

This  report  presents  the  results  of  a  systems  engineering  analysis  of  the 
training  development  process  required  to  produce  an  Integrated  Training  System 
(ITS)  for  a  New  Weapons  System  (NWS) .  This  work  was  performed  in  response  to 
the  findings  of  the  1985  Army  Science  Board  and  the  latest  Army  Aviation 
Mission  Area  Analysis,  which  suggest  that  Army  training  development  for  NWS 
can  be  improved  by  using  a  "top-down"  approach  to  training  system  development 
starting  very  early  in  the  NWS  acquisition  process.  The  report  contains  the 
design  architecture  for  an  Army  ITS  Decision  Support  System  (DSS) ,  portrayed 
in  IDEF  diagrams,  plus  supporting  data  on  potential  users  of  the  system, 
embedded  training,  ITS  data  sources,  and  a  set  of  training  system  design 
elements.  The  architecture  spans  the  full  cycle  of  NWS  development  from 
Milestone  0  through  fielding  and  long  term  maintenance.  It  integrates  all 
aspects  of  ITS  development  within  and  across  Military  Occupational 
Specialties,  such  as  individual,  collective,  and  combined  arms  training  and 
institutional,  unit,  and  distributed  training.  The  architecture  provides  the 
basis  for  the  development  of  the  functional  specifications  for  the  DSS. 
Specific  intervention  points  are  identified  to  provide  the  focus  for  future 
efforts  to  develop  individual  methods  and  procedures  which  are  compatible  with 
the  overall  model. 
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TRAINING  REQUIREMENTS  DETERMINATION,  AND  DESIGN  AND  DEVELOPMENT  METHODS 

Analysis  Msthods 

049  Fischl,  M.A.,  &  Siegel,  A.L.  (1970).  Skill  and  knowledge  requirements  for 
the  AN/BQR-2  DIMUS . (APS  Technical  Report) .  Wayne,  PA:  Applied 
Psychological  Services,  Inc. 

Skill  and  knowledge  requirements  were  determined  for  use  in  developing 
personnel  and  training  requirements  studies  and  to  provide  a  base  line  for  use 
by  Navy  sponsored  definition  evaluation  panels.  Multidimensional  scaling 
analysis  was  employed  to  obtain  operation  and  maintenance  skills  and 
knowledges  for  two  competing  DIMUS  systems.  The  DIMUS  system  is  a  new  digital 
multibeam  steering  sonar  system.  The  DIMUS  skill  and  knowledge  requirements 
were  represented  by  8  factors:  the  maintenance  skill  and  knowledge 
requirements  by  5  factors,  (sic) 

085  Hoyt,  W.G.  (1972) .  System  engineering  of  training  for  a  new  major  system: 
Applied,  computerized,  and  utilized.  Proceedings  of  the  Human  Factors 
Society  -  16th  Annual  Meeting. 

The  subject  of  this  paper  is  a  unique  application  of  system  engineering  of 
training  for  a  new  system  development.  It  is  a  report  of  what  has  been 
accomplished,  and  it  can  serve  as  a  model  for  conducting  similar  projects.  Of 
special  interest  in  this  report  are:  the  approach,  which  is  different  because 
we  have  been  dealing  with  a  new  system  not  yet  operational  and  not  much  is 
contained  in  the  literature  about  new  systems;  but  also  new  solutions  have 
been  found  to  old  problems  and  these  solutions  are  applicable  to  both  old  and 
new  systems.  The  solutions  result  from  good  system  analysis;  adherence  to  the 
principle  of  letting  the  data  drive  the  methodology  rather  than  the 
methodology  driving  the  data;  the  principle  of  developing  a  sufficient  base 
within  each  step  in  the  systems  engineering  process  before  making  decisions 
further  down  the  line,  e.g.,  establishing  the  training  requirements  first, 
prior  to  the  consideration  of  constraints  which  may  be  imposed;  and  the 
principle  of  adequate  record  keeping  and  organization  of  data.  The  latter 
makes  possible  an  easy,  cost-effective  revision,  evaluation  and  reporting 
process.  This  facilitates  understanding,  evaluation,  and  use  of  the  results 
and  consequent  user  acceptance.  Cost-effective  computer  capability  is  used 
throughout  the  system  engineering  process  to  provide  the  visibility  and 
manipulation  necessary  for  the  communication,  control,  and  reporting  process. 

072  Hansen,  D.H.  (1973).  The  analysis  and  development  of  an  adaptive 

instructional  model (a)  for  individualized  technical  training: _ EhflSfi— I 

(U.S.  AFHRL  Technical  Report  No.  72-50(1)).  Brooks  Air  Force  Base,  TX: 
U.S.  Air  Force  Human  Resources  Laboratory. 

Describes  the  purpose  and  function  of  Adaptive  Instructional  Models  (AIM) ,  a 
series  of  computerized  instructional  tasks  for  individual  training  (e.g., 
drill-and-practice,  concept  acquisition,  complex  tutorial,  and  algorithmic 
regression  models) .  Seven  AIMS  are  analyzed  and  recommendations  for  their 
application  in  Air  Force  technical  training  courses  are  presented. 
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134  Rigney,  J.W.,  Morrison,  D.K.,  £  Williams,  L.A.  <1974).  i 
application  of  performance-structure  oriented  CAI  in  nava. 


working  paper  (NAVTRAEQUIPCEN  Technical  Report  No. 
FL:  U.S.  Navy  Training  Equipment  Center. 


73-C-0065-1) .  Orlando, 


Describes  considerations  and  procedures  for  applying  Performance-Structure 
Oriented  Con$>uter-Assisted  Instruction  (CAI)  in  naval  training  in  terms  of  a 
general  diagram  of  the  necessary  elements  in  a  CAI  system.  The  developmental 
steps,  from  job  task-structure  analyses  to  computer  programs  are  described. 

154  Siegel,  A.L.,  Federman,  P.J.,  £  Sellman,  W.S.  (1974).  A  survey  of 
student  measurement  and  course  evaluation  procedures  within  the  Air 
Training  Command.  Lowry  Air  Force  Base,  CO:  Technical  Training  Division. 

Surveyed  the  reactions  of  training-evaluation,  student -measurement,  and 
training-manager-instructor  personnel  to  the  U.S.  Air  Force  Instructional 
System  Development  (ISD)  system.  Favorable  user  reactions,  problems  associated 
with  ISD  procedures,  and  recommendations  for  ISD  program  improvements  are 
presented. 

012  Braby,  R.,  Henry,  J.M.,  Parrish,  W.F.,  £  Swope,  W.M.  (1975).  A  technique 
for  choosing  cost-effective  instructional  delivery  systems  (TAEG  Report 
No.  16) .  Orlando,  FL:  Training  Analysis  and  Evaluation  Group. 

Describes  a  technique  for  choosing  cost-effective  instructional  delivery 
systems  for  proposed  training  programs-the  Training  Effectiveness,  Cost 
Effectiveness  Prediction  (TECEP)  technique.  It  provides  an  orderly  approach 
for  the  skilled  training  system  designer  to  use  in  making  delivery  system 
choices  during  the  conceptual  design  phase.  A  3-step  procedure  is  described 
in  which  training  objectives  are  classified  and  organized  into  groups, 
appropriate  learning  strategies  are  defined  for  each  group,  media  capable  of 
supporting  these  strategies  are  identified,  and  the  costs  of  alternative  forms 
of  training  are  projected.  With  this  information,  optimum  delivery  system 
choices  can  be  made.  A  list  of  12  types  of  learning  algorithms  and  the  class 
of  learning  objectives  each  supports,  separate  tables  for  choosing 
instructional  delivery  systems  for  each  algorithm,  and  a  cost  model  for 
comparing  the  value  of  resources  required  by  alternative  delivery  systems  are 
presented.  A  Fortran  IV  program  listing  of  the  cost  model  is  included. 

013  Branson,  R.K.,  Rayner,  G.T.,  Cox,  J.L.,  Furman,  J.P.,  King,  F.J.  (1975). 

Interaervlce  procedures  for  instructional  systems  development. _ Executive 

summary  and  model.  Tallahassee,  FL:  Florida  State  University, 

Tallahassee  Center  for  Educational  Technology. 

The  report  is  a  five  volume  set  of  procedures  developed  for  the  preparation  of 
a  curriculum  when  interservice  training  is  called  for.  The  procedures  address 
five  major  phases,  which  are:  Analyze,  Design,  Develop,  Implement,  and 
Control.  The  procedures  begin  with  a  methodology  for  conducting  a  job 
analysis  for  the  curriculum  subject  area  for  which  the  instruction  is  to  be 
developed  and  then  to  go  through  18  additional  steps  suitable  for  the 
empirical  development  of  interservice  training.  This  volume  contains  a 
summary  and  model. 
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Other  common  citation  for  the  above  document: 


Department  of  the  Army  (1975) .  Instructional  systems  development 
(TRADOC  Pamphlet  350-30,  five  volumes).  Fort  Monroe,  VA:  U.S.  Army 
Training  and  Doctrine  Command. 

030  Cream,  B.W.,  Eggemeier,  F.T.,  &  Klein,  G.A.  (1975).  Behavioral  data  in 
the  design  of  aircrew  training  devices.  Proceedings  of  the  Human  Factors 
Society  -  19th  Annual  Meeting. 

Attention  has  recently  been  focused  on  the  use  of  ISD  behavioral  data  for 
specifying  aircrew  training  requirements.  However,  behavioral  data  are  not 
sufficient  for  the  actual  specification  of  design  of  equipment,  which 
frequently  represents  the  major  dollar  investment  for  training  programs.  The 
paper  presents  a  methodology  for  designing  training  equipment.  This 
methodology  in  some  ways  goes  beyond  the  collection  of  behavioral  data,  and  in 
other  ways  avoids  the  weakness  of  the  behavioral  data  approach.  The  emphasis 
is  on  ensuring  that  device  fidelity  requirements  are  specifically  correlated 
with  training  requirements.  Four  critical  areas  are  addressed:  (1) 
acquisition  of  behavioral  data;  (2)  determination  of  training  capabilities; 

(3)  performance  measurement;  and  (4)  special  requirements  for  crew 
coordination  training. 

167  Sugarman,  R.C.,  Johnson,  S.L.,  Hinton,  W.M.,  £  Buckenmaier,  C.C.  (1975). 
SAT  revisited-  a  critical  post-examination  of  the  systems  approach  to 
training.  Proceedings  of  the  Human  Factors  Society-1 9th  Annual  Meeting. 

A  unique  application  of  the  Systems  Approach  to  Training  (SAT)  was  carried  out 
for  the  design  of  the  B-l  aircrew  instructional  system.  A  preliminary  report 
was  presented  at  the  1975  Human  Factors  Society  Meeting  by  Johnson,  et  al., 
(1974)  and  Sugarman,  et  al.,  (1974).  Based  on  the  additional  experience,  this 
presentation  will  share  with  the  training  community  a  look  at  some  of  the 
strengths  and  weaknesses  of  the  SAT  process.  The  particular  application 
provided  a  harsh  test  of  the  process  since  the  B-l  air  vehicle  is  still  in  the 
developmental  stages.  This  resulted  in  many  opportunities  to  be  faced  with 
missing  or  incomplete  data  whose  criticality  to  the  analysis  became  more  than 
apparent.  The  general  state-of-knowledge  within  Instructional  System 
Development  (ISD)  itself  is  incomplete,  which  also  became  apparent  when 
applying  a  highly  structured  (i.e.,  internally  consistent  and  documented)  SAT 
process.  The  major  strength  of  the  SAT  process  is  the  capability  it  provides 
for  the  justification  of  many  decisions  that  were  hitherto  prone  to  hand- 
waving.  This  particular  strength  comes  from  the  internal  documentation  for 
each  decision  process  which  is  an  important  feature  of  SAT.  The  adequacy  of 
the  assumptions  underlying  the  decision  processes  then  becomes  subject  to 
scrutiny.  The  power  of  the  documentation  is  enhanced  by  the  use  of  computer 
aids  in  data  management  and  economic  analysis  which  facilitate  the 
documentation.  Within  the  present  paper,  problem  areas  which  weaken  the  SAT 
process  are  also  discussed.  These  include  the  data  bases  for  the  task 
analysis  and  economic/policy  data,  the  media  selection  process,  and  the 
development  of  instructional  strategies. 
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001  Aagard,  j.a.,  &  Braby,  R.  (1976).  Learning  guide  line  a  and  algorithms  for 
types  of  training  objectives  (TAEG  Report  No.  23) .  Orlando,  FL:  Training 
Analysis  &  Evaluation  Group. 

Presents  training  strategies  for  11  common  classes  of  objectives:  recalling 
bodies  of  knowledge,  using  verbal  information,  rule  learning  and  using, 
decision  making,  detecting,  classifying,  and  identifying  symbols,  voice 
communicating,  recalling  procedures  and  positioning  movements,  steering  and 
guiding  and  continuous  movement,  and  performing  gross  motor  skills.  The 
strategy  for  each  class  of  instructional  objectives  is  made  up  of  3  parts . 

The  first  is  a  definition  of  the  class  and  a  description  of  the  uniqueness  of 
each  class.  The  second  is  a  set  of  learning  guidelines,  consisting  of  a 
series  of  statements  which  prescribe  specific  learning  elements  to  be  built 
into  the  design  of  a  training  system.  These  guidelines  are  based  mostly  on 
learning  theory  and  on  practical  experience  and  represent  information 
available  for  prescribing  general  solutions  for  a  class  of  training  problems. 
The  third  part  is  a  learning  algorithm,  expressed  as  a  flow  chart  of  a 
sequence  of  learning  events .  It  represents  a  logical  arrangement  of  events 
called  for  in  the  guidelines.  These  guidelines  and  algorithms  may  be  used  by 
training  system  designers  as  guidance  in  specifying  learning  events  and 
activities,  selecting  instructional  delivery  vehicles,  designing  instructional 
materials,  evaluating  existing  instructional  materials,  and  recording  field 
experience  for  use  in  improving  guidelines  and  algorithms. 

038  Duffy,  L.R.,  Miller,  R.B.,  &  Staley,  J.D.  (1977).  Design  of  training 
systems;  Computerization  of  the  educational  technology  assessment  model 
/ETAM)  (Training  Analysis  and  Evaluation  Group  Report  No.  40(1)).  Orlando, 
FL:  Training  Analysis  and  Evaluation  Group. 

The  first  volume  of  a  two-volume  report  summarizes  the  analysis,  design,  and 
development  activities  associated  with  the  Educational  Technology  Assessment 
Model  (ETAM)  .  It  contains  relevant  background  information  and  results  of 
prior  studies  leading  to  the  finalized  ETAM  procedures  and  computerized 
routines.  A  comparison  of  the  manual  and  automated  approach  is  included,  and 
data  base  structures  and  ETAM  program  flow  are  described.  The  second  volume 
(see  PA,  Vol  59:11358)  contains  the  344-page  appendix. 

039Duffy,  L.R.,  Miller,  R.B.,  &  Staley,  J.D.  (1977).  Design  of  training 

systems ■  Computerization  of  the  educational  technology  assessment  model. 
/ETAM) :  II  (Training  Analysis  and  Evaluation  Group  (TAEG)  Report,  40(2)). 
Orlando,  FL:  Training  Analysis  and  Evaluation  Group. 

The  second  volume  of  a  2-volume  report  (for  the  first  volume,  see  PA,  Vol 
59:13361)  on  the  Educational  Technology  Assessment  Model  (ETAM)  contains  the 
appendix  which  includes  the  results  of  a  study  on  the  indexing  of  innovations 
and  the  assignment  of  taxonomic  descriptors  to  courses,  job/tasks,  and 
instructional  vehicles.  The  appendix  also  includes  program  documentation  on 
the  ETAM  Range-of -Effect,  4  pages  of  bibliographic  references,  and  additional 
information  supporting  the  ETAM  design. 

011  Booher,  H.R.  (1978).  Job  performance  aiding  systems  technology  selection 
algorithm.  Proceedings  of  the  Human  Factors  Soclety-22nd  Annual  Meeting. 
158-162. 

A  nine-step  selection  algorithm  is  described  which  relates  personnel  aptitude, 
experience,  task  and  equipment  complexity,  and  degree  of  proceduralization  in 
a  decision  chain  for  choosing  optimum  combinations  of  job  performance  aids 
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(JPAs)  and  training  technologies.  The  algorithm  has  been  useful  in  (1) 
identifying  JPA  research  and  technology  gaps,  (2)  identifying  candidate  JPA 
systems  for  Navy  personnel,  maintenance  and  training  trade-off  analyses,  and 
(3)  in  selecting  specific  JPA  systems  for  test  and  evaluation  in  the  Navy 
Performance  Aids  Test  and  Evaluation  project.  The  algorithm  development  has 
also  included  efforts  to  identify  and  evaluate  specifications  and  guidelines 
available  for  JPA  procurement /implementation  and  to  highlight  areas  where 
information  should  be  gathered  for  an  integrated  JPA/training/maintenance/ 
personnel  user  handbook. 

055  Gibbons,  A.S.  £  Hughes,  J.A.  (1978).  A  method  of  deriving  hierarchies  of 
instructional  objectives .  Proceedings  of  the  Human  Factors  Society-22nd 
Annual  Meeting.  88-90. 


A  method  is  outlined  for  the  development  of  hierarchies  of  instructional 
objectives  to  be  used  in  instructional  development  or  training.  The  method 
proposed  increases  the  specificity  normally  encountered  for  hierarchy 
development  and  allows  participation  of  non-scientist  level  workers  in  the 
generation  of  analyses  with  anticipated  logical  qualities.  A  system  of 
deriving  hierarchy  patterns  growing  out  of  the  application  of  the  analysis  is 
also  described  which  assists  the  development  and  simplifies  the  analysis 
process . 

143  Sauer,  D.N. ,  £  Askren,  W.B.  (1978).  Validation  of  an  expert  est  imate 

technique  for  prediction  manpower,  maintenance,  and  training  requirements 
for  proposed  Air  Force  systems  (AFHRL-TR-78-19) .  Brooks  Air  Force  Base, 

TX:  U.S.  Air  Force  Human  Resources  Laboratory. 

The  objectives  were  to  determine  the  validity  of  an  expert  estimate  technique 
for  predicting  manpower,  maintenance,  and  training  requirements  for  equipment 
in  the  early  stages  of  design,  and  to  develop  a  guide  for  implementing  the 
technique.  Sixty  Air  Force  technicians  from  two  avionics  AFSCs  participated  as 
expert  estimators.  Twenty  of  these  technicians  were  also  qualified  avionics 
instructors.  The  €0  technicians  made  estimates  of  manpower,  maintenance,  and 
training  requirements  using  only  an  engineering  description  of  an  operational 
avionics  component .  The  description  contained  information  available  during 
the  early  design  stages  of  the  component.  The  accuracy  of  the  estimates  was 
determined  by  comparing  them  with  manpower,  maintenance,  and  training  data 
associated  with  the  operational  equipment.  The  results  indicate  that 
maintenance  task  time,  crew  size,  skill  level,  career  field,  and  task 
difficulty  can  be  estimated  with  a  satisfactory  degree  of  accuracy.  Training 
times  were  greatly  overestimated.  The  estimates  of  required  training 
facilities/equipment  and  the  inpact  of  design  features  on  maintenance  were 
nonconclusive .  The  results  also  indicate  that  ten  technicians  with  skill 
level  5  and  with  system  experience  on  equipment  similar  to  the  proposed 
equipment  will  produce  acceptable  estimates.  A  prototype  guide  for  using  the 
technique  was  prepared. 

057  Gilbert,  A.C.,  Waldkoetter,  R.O.,  Raney,  J.L.,  £  Hawkins,  H.H.  (1980). 
Efficacy  of  a  training  priorities  model  in  an  Army  environment .  Catalog 
al  Selected  Bocuaenta  in  Psychology,  1&,  69-70. 


14 


108  Lenzycki,  H.P.,  k  Finley,  D.L.  (1980).  How  to  determine  training  device 

raaMija—Btfl  and  characteristics: _ A  handbook  for  training  develapar.a  <ari 

Research  Product  80-25).  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences. 

This  handbook  was  prepared  for  use  by  U.S.  Army  training  developers  and  others 
accountable  for  determining  requirements  for  and  characteristics  of  training 
devices  to  support  hardware  fielding.  It  contains  a  description  of  a 
methodology  for  performing  the  front-end  analysis  needed  to  determine  what 
tasks  have  to  be  trained  (training  requirements) ;  the  need  for  a  training 
device (s)  as  a  medium  for  developing  required  task  skills  (training  device 
requirements) ;  and  the  characteristics  that  the  device  should  possess  in  order 
to  promote  effective  learning  (training  device  characteristics) .  The  handbook 
supplements  and  extends  the  contents  of  ISD  Pamphlet  350-30  in  that  it:  (1) 
further  details  the  process  for  developing  task  descriptions  and  acquiring 
other  job  information  needed  to  assess  the  demands  imposed  on  the  operator  by 
the  equipment,  operational  environment,  other  crew  members  and  interactive 
systems;  (2)  provides  additional  criteria  and  rating  scales  for  use  in  the 
task  analysis  process;  (3)  allows  for  specific  identification  of  requirements 
for  training  equipment  (actual  or  device)  as  opposed  to  the  ISD  process  which 
leads  to  the  identification  and  selection  of  generic  training  media 
alternatives;  and  (4)  leads  to  the  identification  of  specific  characteristics 
of  devices  needed  to  provide  the  essential  skills  training. 

174  Vineberg,  R.,  k  Joyner,  J.N.  (1980).  Instructional  system  development 
(ISD)  in  the  armed  services:  Methodology  and  application  (ARI:  HumRRO-TR- 
80-1) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

The  study  examined  Instructional  System  Development  (ISD)  methodologies  and 
practices  in  the  Army,  Navy,  Marine  Corps,  and  Air  Force  during  Aug.  1977- 
Mar.  1979.  Findings  are  based  on  (1)  analysis  of  the  primary  guidance 
documents  used  in  the  Armed  Services  for  conducting  ISD,  (2)  questionnaire 
survey  of  209  units,  agencies  and  schools  where  training  is  developed  in  the 
Services,  and  (3)  detailed  interview  of  training  developers  at  33 
organizations  to  determine  how  57  courses  were  designed.  Major  findings  are 
that  ISD  is  not  being  used  either  to  optimize  total  system  effectiveness  or  to 
maximize  training  efficiency. 

The  iterative  and  derivative  character  of  the  methodology  can  insure  that 
training  will  be  relevant  to  job  requirements  if  its  procedures  are  faithfully 
carried  out.  In  practice,  however,  many  of  the  components  of  ISD  are  omitted 
and  the  close  connection  between  components  to  make  the  process  truly 
derivative  is  not  maintained.  Thus  the  potential  of  ISD  to  insure  that 
training  meets  job  requirements  is  not  being  realized.  The  conception  of  ISD 
that  is  most  adequately  represented  in  current  applications  of  ISD  is  as  the 
use  of  specific  elements  of  modern  training  technology,  i.e.,  job  analysis, 
self-paced  instruction.  Considerable  evidence  leads  to  the  conclusion  that 
although  the  generation  of  the  products  of  ISD  can  be  mandated,  the  ISD 
process  itself  cannot.  Training  is  both  developed  and  evaluated  within  the 
training  subsystem,  whereas  the  consequences  of  training  occur  in  operational 
units.  It  is  recommended  that  operational  commands  be  given  a  larger  role  in 
identifying  job  requirements,  establishing  training  requirements,  and 
evaluating  the  performance  of  training  graduates.  The  report  also  contains 
findings  and  recommendations  for  19  specific  steps  of  the  ISD  process. 
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Hawley,  J.K.,  &  Dawdy,  E.D.  (1981).  ! 

HaaaaaaHPt  capability  ( DORAC  1  : _ Feasibility  and  utility  (Army  Research 

Institute:  Applied  Sciences  Associates  Final  Report) .  Alexandria,  VA: 

U.S .  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

This  report  presents  the  results  of  an  investigation  of  the  feasibility  and 
utility  of  implementing  the  Training  Device  Operational  Readiness  Assessment 
Capability  (DORAC)  concept  throughout  the  Army.  The  investigation  addresses 
three  separate  aspects  of  feasibility. 

Another  aspect  of  DORAC  concept  feasibility  concerns  the  ability  to  select  the 
most  cost-effective  proficiency  assessment  capability  from  among  a  range  of 
alternatives.  To  this  end.  Cost  and  Information  Effectiveness  Analysis  (CIEA) 
methodology  based  on  multi-attribute  utility  measurement  (MAUM)  was  developed 
and  is  presented  in  a  companion  report.  In  this  report,  the  MAUM-CIEA 
methodology  is  demonstrated  using  a  set  of  hypothetical  DORACs  for  the  M16A1 
rifle. 

The  final  section  of  the  report  integrates  the  feasibility  results  into  a 
series  of  recommendations  for  optimizing  the  payoff  from  a  DORAC,  and  suggests 
a  series  of  steps  for  further  investigation  of  the  concept  and  related 
technologies . 

127  Pieper,  W.J.,  Guard,  N.R.,  Michael,  W.T.,  &  Kordak,  R.S.  (1981) . 

Training  developers  decision  aid  for  optimizing  performance-based  training 
in  machine  ascendant  MOS.  Catalog  of  Selected  Documents  In  Psychology. 

11,  87. 

130  Reece,  H.F .  (1982) .  Computers,  learning  styles  and  instructional 
materials-  are  they  related?  Academic  Therapy.  lfi.(2),  157-161. 

Describes  a  microcomputer-based  system  for  organizing  instructional  materials 
in  a  school  system  so  that  the  teacher  can  select  appropriate  materials  for 
special  students.  Steps  in  designing  the  system  include  reviewing  resource 
material  in  terms  of  educational  goals,  grade  levels,  and  instructional 
strategies;  establishing  a  data  file;  designing  a  program  that  allows  teachers 
and  other  staff  to  perform  custom  computer  searches  of  the  database  to  see 
what  material  in  the  system  are  appropriate  for  a  given  student;  and 
evaluating  the  results. 

162  Spears,  m.d.  (1983).  gxQcaaaea  of  skill  performance; _ a  foundation  for 

the  design  and  use  of  training  equipment  (NAVTRAEQUIPCEN  Technical  Report 
78-C-0113-4) .  Orlando,  FL:  U.S.  Navy  Training  Equipment  Center. 

Discusses  analysis  of  skill  performance  as  a  basis  for  the  design  of  low-cost 
training  devices  for  use  in  the  military.  Cognitive  motor  skills  are  analyzed 
in  terms  of  the  processing  of  information.  Cognitive  processes  involved  in 
motor  skills  and  information  processing  include  task  recognition;  task 
cosqprehension;  goal  setting;  planning  performance;  initiation,  monitoring,  and 
regulation  of  performance,  stimulus  encoding  and  elaboration,  attentional 
processes,  retention  and  retrieval  of  information;  hierarchical  schemata  for 
discrimination  and  generalization;  motivation,  and  skill  integration  and 
automation.  For  motor  skills,  emphasis  is  placed  on  structural  characteris¬ 
tics  of  movements,  temporal  characteristics  of  movements,  signal  discrimina¬ 
tion  and  generalizations,  roles  of  sensory  modes  and  their  interactions,  and 
patterns  of  skill  integration.  Empirically  based  concepts  provide  an 
operational  means  of  manipulating  variables  during  training,  and  examples  are 
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presented  of  methods  for  empirically  assessing  roles  of  various  processes .  It 
is  concluded  that  analyses  could  be  extrapolated  to  general  training 
technology  and  to  the  design  of  training  devices. 

007  Ball,  F.M.  (1984) .  The  instructional  systems  development  model  and 

systems  engineering.  Proceedings  of  the  Human  Factors  Societv-28th  Annual 
Meeting.  845-849. 

Two  implicit  perspectives  governing  the  organization  of  human  performance  have 
evolved  within  the  training  community,  leading  to  two  different  approaches  to 
the  systematic  development  of  performance-based  training.  These  are 
identified  here  as  the  Instructional  Systems  Development  (ISD)  approach,  and 
the  Systems  Engineering  approach.  Both  employ  similar  procedures  with  the 
exception  that  the  ISD  approach  relies  on  an  initial  job-oriented  analysis; 
the  Systems  Engineering  approach  relies  on  a  functional  systems  analysis. 

This  difference  affects  the  scope  and  organization  of  the  data.  Resulting 
training  programs  and  materials  are  more  suited  to  some  applications  than  to 
others.  This  paper  presents  a  conceptual  framework  illustrating  the 
relationship  and  argues  that  the  two  approaches  are  theoretically  equivalent. 
Within  this  framework  practical  training  development  implications,  problems, 
and  benefits  are  discussed. 

078  Hawley,  J.K.  (1984).  Some  considerations  in  the  design  and 

implementation  of  a  training  device  performance  assessment  capability. 
Proceedings  of  the  Human  Factors  Society-2Bth  Annual  Meeting.  201-205. 

In  an  era  of  increasing  system  complexity  and  resource  constraints,  live-fire 
exercises  on  a  scale  necessary  to  insure  personnel  proficiency  are  not 
feasible  for  many  weapons  systems.  A  partial  solution  to  the  problem  of 
insufficient  performance  evaluation  involves  the  use  of  training  devices  in 
lieu  of  operational  equipment  in  the  conduct  of  evaluation  exercises.  A 
trainer  that  has  been  extended  to  include  an  embedded  performance  evaluation 
capability  is  referred  to  as  a  training  device  performance  assessment 
capability  (DPAC) .  The  present  paper  presents  a  series  of  guidelines  for  the 
design  and  development  of  DPACs.  The  guidelines  were  prepared  on  the  basis  of 
background  research  and  actual  experience  in  developing  a  DPAC  for  the  Patriot 
air  defense  missile  system.  Because  of  their  association  with  an  actual 
developmental  effort,  the  guidelines  have  considerable  practical  utility  for 
DPAC  design  and  in^lementation.  A  number  of  issues  inportant  for  the  full 
realization  of  DPAC  potential  also  are  discussed. 

091  Johnson,  S.L.  (1985).  Using  mathematical  models  of  the  learning  curve  in 
training  system  design.  Proceedings  of  the  Human  Factors  Society-  29th 
Annual  Meeting. 

This  paper  reviews  and  synthesizes  some  of  the  literature  on  the  mathematical 
modeling  of  the  learning  curve  that  has  been  developed  by  many  diverse 
technical  disciplines.  These  disciplines  include  the  behavioral  sciences, 
manufacturing  engineering,  and  accounting.  The  paper  discusses  the 
relationships  among  the  models  and  the  potential  uses  of  the  models  in  the 
design,  development,  and  control  of  training  systems. 
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099  Kemmer-Richardson,  S.,  Lamos,  J.P.,  £  West,  A.S.  (1985).  Computer- 
assisted  instruction:  Decision  handbook  (U.S.  AFHRL  Tech  Rpt  84-46)  . 
Brooks  Air  Force  Base,  TX:  U.S.  Air  Force  Human  Resources  Laboratory. 

Notes  that  this  handbook  is  meant  to  serve  as  a  resource  and  reference 
guide  for  U.S.  Air  Force  instructional  managers  who  are  considering  the 
adoption  or  expansion  of  computer-assisted  instruction  (CAI)  at  some  future 
time  and  also  as  a  decision  aid  for  an  instructional  manager  currently 
involved  in  the  CAX  decision  process.  The  use  of  the  handbook  is  considered 
to  be  appropriate  for  considering  the  implementation  of  CAI  in  a  new  course  or 
in  an  existing  course.  The  workbook  portion  of  the  handbook  presents  specific 
tools,  in  the  form  of  worksheets,  for  evaluating  the  need  for  CAI,  for 
identifying  configurations  most  closely  matched  to  instructional  needs  and 
practices,  and  for  estimating  the  feasibility  of  initiating  CAI 
implementation . 

123  Myers,  G.L.,  £  Fisk,  A.D .  (1985).  Toward  a  skill  based  training 

technology:  review,  evaluation,  and  application  of  automatic/controlled 
processing  training  guidelines.  Proceedings  of  the  Human  Factors  Society- 
29th  Annual  Meeting. 

In  the  present  paper  it  is  argued  that  there  are  two  major  purposes  of 
instruction  for  skill  acquisition.  The  first  is  to  insure  that  the  consistent 
components  of  the  skill  are  identified  by  the  trainee.  The  second  purpose  is 
to  provide  many  correct  executions  of  (or  exposures  to)  these  consistent 
components .  A  brief  review  of  the  laboratory  research  supporting  the  above 
statements  is  provided.  The  concept  of  consistent  component  training  is 
described,  and  a  preliminary  application  of  the  non-laboratory  approach  is 
outlined.  Guidelines  are  discussed  for  specifying  consistent  trainable 
components  of  a  skill  or  task. 

010  Bloom,  R.S.,  £  Yanko,  D.M.  (1986) .  Use  of  job  task  analysis  to  determine 
training  requirements.  Proceedings  of  the  Human  Factors  Society,  1448- 
1451. 

This  paper  contains  a  discussion  of  how  two  job  task/human  performance 
analysis  systems  have  been  used  in  the  nuclear  utility  industry  to  a) 
determine  training  program  curriculum  and  content  requirements,  and  b)  provide 
data  for  management  decisions  regarding  training  system  development.  This  is 
a  presentation  of  the  methods  used  in  a  performance  analysis  of  a  variety  of 
workers,  the  products  created,  and  to  what  end  those  products  were  used.  The 
paper  is  a  pragmatic  application  which  training  systems  personnel  can  use  in 
their  work  situations  to  develop  training  programs  grounded  in  data  generated 
by  job  task/human  performance  analysis. 

118  Montague,  w.E.  (1986) .  Technical  training  and  Instructional  Systems 
Development:  Can  a  cognitive  model  help?  Advances  in  Reading /Language 

Research,  1,  119-138. 

Discusses  deficiencies  in  the  Navy' s  approach  to  Instructional  Systems 
Development  (ISD)  for  technical  training  and  suggests  improvements .  Navy 
training  is  not  systematically  organized,  focuses  on  preparatory  training,  is 
based  toward  text  instruction  and  against  contextual  relations,  and  has  a  weak 
link  between  jobs  and  training,  while  the  ISD  process  was  developed  to 
strengthen  the  connection  between  training  and  the  work  environment  and  to 
permit  management  by  personnel  untrained  in  teaching  and  related  specialties, 
it  has  not  worked  well.  The  overly  simple  behavioral  psychoeducational  model 
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adopted  by  ISD  fails  to  incorporate  recent  developments  in  cognitive  research 
on  mental  representations  and  analytic  procedures.  A  practical  mechanism  for 
improvement  is  to  automate  as  much  of  the  ISD  process  as  possible,  in  a  way 
that  is  unobtrusive  to  users  and  provides  them  with  the  capabilities  needed 
for  developing  and  delivering  training. 

155  Singer,  M.J.,  &  Perez,  P..S.  (1986).  A  demonstration  of  an  expert  system 
for  training  device  design.  Journal  of  Computer-Based  Instruction.  12(2), 
58-61. 

Describes  the  conceptual  development  of  a  decision  support  system  for  aiding 
in  the  design  of  training  devices.  The  system  applies  expert  system 
technology  and  utilizes  a  production  rule  approach.  It  is  noted  that  the 
primary  use  of  a  training  device  or  simulator  is  to  provide  trainees  with  the 
conditions,  characteristics,  and  events  necessary  for  the  acquisition  of 
skills  that  will  be  performed  on  the  job. 

059  Goldberg,  I.,  &  Khattri,  N.  (1987).  A  review  of  models  of  cost  and 

training  effectiveness  analysis  (CTEA)  .  Volume  I: _ Training  fef fectiveneaa 

analysis  (ARI  Research  Note  87-58) .  Alexandria,  VA:  O.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

This  research  note  reviews  the  formal  predictive  and  prescriptive  models  of 
training  effectiveness  analysis  and  related  Army  guidance.  Most  of  the  models 
are  concerned  with  the  analytic  formation  of  entire  training  programs  early  in 
the  weapons  systems  acquisition  process.  Some  models  also  include  manpower, 
personnel,  and  human  factors  considerations.  A  number  of  models  focus  on  the 
formulation  of  training  devices  and  simulators.  Computer  Based  Instruction 
(CBI)  has  not  often  been  included  in  early  formation  of  training  programs  for 
new  weapons  systems.  Possible  reasons  for  this  finding  are  discussed.  Most 
of  the  training  program  models  also  include  a  cost  model,  but  the  cost  models 
are  not  necessarily  adequate.  The  models  concerned  with  training  devices  and 
simulators  do  not  have  associated  cost  models.  The  lessons  learned  included 
the  following:  1)  there  are  many  useful  models  available  for  the  formulation 
of  training  programs  early  in  the  weapons  systems  acquisition  process;  2)  the 
validity  of  these  models  needs  to  be  tested,  recommendations  are  given  for 
comparative  validity  studies;  3)  models  for  training  devices  and  simulators 
appear  to  need  further  development,  related  cost  models,  and  validation;  4) 
further  attention  needs  to  be  given  to  formal  CTEA  models  for  advanced  phases 
of  the  weapons  system  acquisition  process,  and  for  non-system  training. 

142  Ryder,  J.M.,  Redding,  R.E.,  6  Bechschi,  P.F.  (1987).  Training 

development  for  complex  cognitive  tasks.  Proceedings  of  the  Human  Factors 
Society  -  31st  Annual  Meeting.  1261-1265. 


This  study  evaluated  current  training  methodologies,  particularly 
Instructional  Systems  Development  (ISD),  and  recent  developments  in  cognitive 
science  to  determine  how  training  procedures  should  be  modified  to  support 
training  for  tasks  which  require  complex  cognitive  skills.  He  contend  that 
ISD  is  still  viable  if  procedures  are  developed  for  the  training  of  cognitive 
skills.  An  important  component  of  ISD  which  needs  to  be  modified  to  support 
training  of  cognitive  skills  is  the  task  analysis.  He  discuss  the  need  for 
integrating  efficient  and  cost-effective  cognitive  task  analysis  methodologies 
with  traditional  analysis  methods. 


158  Sing«r,  M.J.,  k  Sticha,  P.J.  (1987).  Designing  training  devices:  the 

optimisation  of  simulation-based  training  systems.  Proceedings  of  the  9th 
Intaraarvlca/Ioduatry  Training  Syatsma  Conference,  26-31. 

Effective  training  devices  are  those  that  meet  training  requirements  at 
minimum  cost,  or  provide  the  maximum  training  benefit  for  a  given  cost.  The 
Optimisation  of  Simulation-Based  Training  Systems  (OSBATS)  is  a  model  that  is 
designed  to  facilitate  the  investigation  of  trade-offs  involved  in  developing 
effective  training  device  concepts.  The  model  is  based  on  benefit  and  cost 
approximations  that  are  used  to  analyze  trade-offs  between  various  training 
device  features  in  developing  a  device  configuration,  and  then  conducts 
similar  trade-offs  between  different  training  device  configurations.  The 
development  of  OSBATS  has  been  more  theoretical  than  the  typical  decision 
support  system  or  aid,  but  shares  many  of  the  attributes  of  the  standard 
decision  aid.  The  tools  or  modules  that  comprise  the  model  address  the 
following  activities:  a)  the  clustering  of  tasks  for  developing  coherent 
training  device  configurations,  b)  the  identification  of  optimal  instructional 
features  for  a  task  cluster,  c)  the  specification  of  optimal  fidelity  levels 
for  a  task  cluster,  d)  the  selection  of  the  minimum  training  device  family 
that  meets  training  requirements,  and  e)  the  allocation  of  training  resources 
in  the  family  of  suggested  training  devices.  The  final  output  of  the  OSBATS 
model  is  a  functional  description  of  the  optimal  set  of  efficient  training 
devices  given  the  tasks,  training  criteria,  and  cost  constraints. 

160  Solick,  R.E.,  &  Lussier,  J.W.  (1988).  Design  of  battle  simulations  for 
command  and  staff  training  (ARI  Technical  Report  788) .  Alexandria,  VA: 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Describes  design  criteria  (e.g.,  training  objectives,  performance  measurement, 
simulation  requirements)  for  the  next  generation  of  automated  battle 
simulation  based  on  behavioral  research  on  command  and  staff  training. 

158  Smith,  B.R.,  &  Banda,  C.P.  (1989).  Use  of  a  knowledge-based  system  to 
assess  aircrew  training  requirements  as  part  of  conceptual  design. 
Proceedings  of  the  1389  IEEE -International  Conference  on  Systems.  Man,  and 
Cybernetics ■ 

An  important  consideration  in  developing  the  mission  requirements  and  cockpit 
equipment  for  an  evolving  aircraft  is  the  training  needed  to  allow  aircrews  to 
safely  and  effectively  operate  the  completed  vehicle.  As  part  of  a  computer- 
based  cockpit  design  and  analysis  workstation,  a  prototype  training  assessment 
tool  has  been  developed  which  estimates  the  training  resources  and  time 
imposed  by  the  anticipated  mission  and  cockpit  design.  Embedding  instruc¬ 
tional  system  and  training  analysis  domain  knowledge  in  a  production  system 
environment,  this  tool  allows  crew  station  designers  to  readily  determine  the 
training  ramifications  of  their  choices  for  cockpit  equipment,  mission  tasks, 
and  operator  qualifications.  Initial  results  have  been  validated  by 
comparison  to  an  existing  training  program,  demonstrating  the  tool's  utility 
as  a  conceptual  design  aid  and  illuminating  areas  for  future  development. 
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164  Sticha,  P.J.,  Blacksten,  H.R.,  Buede,  D.M.,  Singer,  M.J.,  Gilligan,  E.L., 
Mumaw,  R.J.,  Morrison,  J.E.  (1990).  Optimization  of  simulation-baaed 

training  system: _ Model  description,  implementation,  and  evaluation  (ari 

Technical  Report  896).  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences . 

A  model  for  the  optimization  of  simulation-based  training  systems  was 
developed  using  systematic,  top-down  design  procedure.  The  model  consists  of 
five  tools  that  address  the  following  problems:  a)  determining  which  tasks 
should  be  trained  by  part-mission  devices,  full-mission  simulators,  or  actual 
equipment;  b)  specifying  instructional  features  needed  to  train  a  set  of  tasks 
efficiently;  c)  specifying  the  levels  of  fidelity  that  should  be  provided 
along  several  fidelity  dimensions  in  order  to  meet  task  training  requirements 
and  satisfy  cost  limits;  d)  determining  the  family  of  training  devices  that 
can  train  all  required  tasks  at  minimum  cost;  and  e)  determining  the  optimal 
allocation  of  training  time  to  training  devices,  given  constraints  on  device 
use.  The  tools  share  common  data  on  task  requirements,  training  device 
features,  and  costs. 

A  prototype  decision  support  system  was  developed,  and  a  formative  evaluation 
was  conducted.  The  model  was  demonstrated  on  Army  rotary-wing  aviation  tasks, 
and  specifications  for  application  to  armor  maintenance  were  developed.  The 
report  describes  the  model  using  the  IDEFO  (Integrated  Computer-Aided 
Manufacturing  Definition)  system  modeling  language.  In  addition,  the  model  is 
described  for  an  example  problem  from  a  user's  perspective.  The  example 
problem  illustrates  the  way  that  the  OSBATS  model  organizes  the  decision 
process,  the  kinds  of  analyses  that  can  be  performed,  and  the  kinds  of  output 
that  are  produced.  It  was  concluded  that  the  prototype  provides  an  adequate 
interactive  environment  *.n  which  the  developer  can  perform  several  kinds  of 
trade-off  analyses.  The  OSBATS  software  includes  the  data  necessary  to  use 
the  model  for  certain  problems  in  Army  rotary-wing  aviation. 
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Strategies 


089  Jealous,  F.S.,  Blalek,  H.M.,  Pitpit,  F.,  &  Gordon,  P.  (1975).  Developing 
the  potential  of  low  ability  personnel  (Mo  75-6)  .  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Examined  the  scope  of  the  learning  capacity  o£  marginal  U.S.  Army  personnel, 
the  longitudinal  effects  of  long-term,  self-managed  learning  strategies,  and 
the  proficiency  levels  reached  as  a  result  of  the  application  of  these 
strategies.  Data  were  collected  on  the  self-selected  activities  engaged  in  by 
the  24  participants  and  the  proficiency  levels  reached.  Considerable  gains 
were  reported  in  both  areas  for  a  majority  of  the  participants. 

Recommendations  for  setting  up  a  special  individualized  instruction  program 
are  listed  in  the  event  that  utilization  of  large  numbers  of  marginal 
personnel  is  necessary. 

008  Bell,  C.R.  (1977) .  Criteria  for  selecting  instructional  strategies. 
Training  and  Development  Journal.  11(10),  3-7. 

Examines  systematic  ways  of  deciding  which  instructional  approach  is  best  for 
a  given  training  program.  First,  training  content  must  be  developed  and 
sequenced  on  the  basis  of  needs  of  the  organization.  Environmental  realities 
are  the  other  major  factors  in  training  design  and  are  defined  to  include 
cultural  factors  and  learner  characteristics.  Background  constraints  include 
availability  of  money,  time,  staff,  space,  and  equipment,  with  class  size  also 
to  be  considered.  The  best  prepared  material  goes  for  nothing  if  the  wrong 
strategy  is  used,  and  similarly  an  appropriate  instructional  strategy  cannot 
save  a  weak  curriculum.  Increasing  professionalism  of  the  field  of  human 
resource  development  warrants  increasing  participation  by  training 
professionals  in  the  decision-making  process  utilized  to  determine  both  type 
of  training  and  training  objectives. 

044 Federico,  p.a.  (1978).  Accommodating ..inatructiQn  to  student 

characteristics ; _ Trends  and  issues  (NPRDC  Technical  Report,  No  Tr  79-1) . 

San  Diego,  CA:  U.S.  Navy  Personnel  Research  and  Development  Center. 

Reviews  the  relevant  professional  literature  concerning  adaptive  teaching 
systems.  Several  alternative  approaches  to  accommodating  instruction  to 
student  characteristics  are  identified  and  discussed.  Several  alternative 
approaches  to  accommodating  instruction  to  student  characteristics  are 
identified  and  discussed.  Several  recommendations  are  made  regarding  what 
additional  research  and  development  efforts  are  needed  to  ensure  the 
successful  implementation  of  adaptive  instructional  strategies  in  U.S.  Navy 
training. 

052  Foley,  J.P.  (1979).  Instructional  materials  for  improved  -job  performance 
(U.S.  AFHRL  Technical  Report  No.  78-99) .  Wright-Patterson  Air  Force  Base, 
OH:  U.S.  Air  Force  Human  Resources  Laboratory. 

Discusses  the  applicability  of  technologies  developed  by  the  U.S.  military 
services  for  training  industrial  and  vocational  education  teachers.  Two  types 
of  technologies  are  emphasized:  task  oriented  training  (TOT)  and  improved  task 
guidance  and  information  (ITG&I) .  The  most  task  oriented  ITG&I  are  fully 
proceduralized  job  performance  aids  (FPJPA)-  a  characteristic  that  facilitates 
trade-off  between  FPJPA  and  TOT  sometimes  called  the  "head" /book  trade-off. 

The  key  characteristics  and  requirements  of  the  FPJPA  technology  are  briefly 
described  and  include  a  formal  task  identification  and  analysis  (TI&A)  of  each 
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identified  task,  standard  language  and  keyed  pictorials  for  maintenance 
instructions,  appropriate  format  for  the  maintenance  environment,  and  "hands 
on"  verification  of  the  effectiveness  of  each  FPJPA  product,  utilizing  Ss  from 
the  target  user  population.  The  task  identification  process  is  common  to  both 
TOT  and  FPJPA,  and  the  analysis  process  should  control  the  content  of  TOT, 
FPJPA,  and  TOT/FPJPA  trade-off.  Samples  of  ITG£I,  including  FPJPA,  are 
displayed  but  it  is  cautioned  that  content  of  instructional  materials  placed 
in  this  format  must  be  controlled  by  the  developmental  process.  Industrial 
applications  of  the  ITG£I  and  TOT  technologies  and  implications  for  vocational 
and  industrial  teacher  education  are  described. 

150  Shields,  J.L. ,  Joyce,  R.P.,  &  VanWert,  J.R.  (1979).  Chaparral  skill 
retention  (Research  Report  1205).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

Examined  retention  of  Chaparral  missile  crewmen  skills  to  determine  the  most 
effective  schedule  of  refresher  training.  Six  tasks  included  in  the  Soldier's 
Manual  were  evaluated  using  hands-on  performance  (P)  tasks,  which  were 
completed  using  the  appropriate  technical  manual  (TM) .  99  U.S.  Army  soldiers 

were  divided  into  3  experimental  groups.  All  Ss  were  tested  immediately  after 
completing  Advanced  Individual  Training  (AIT) ,  were  tested  and  trained  to 
standard  on  arrival  at  the  Chaparral  unit,  and  were  retested  4  months  later. 
Groups  1  and  2  were  also  retested  and  trained  1  and  2  months  post  arrival, 
respectively.  Group  3  received  no  additional  testing  or  refresher  training. 
Feedback  given  to  Ss  following  the  first  testing  apparently  provided  training 
so  that  P  improved  over  the  interval  between  AIT  and  first  unit  testing.  P 
declined  slightly  thereafter,  suggesting  that  once  Ss  learned  to  perform  the 
task  to  standard  using  the  TM,  they  continued  to  perform  at  or  near  standard 
over  the  4-month  interval.  It  is  suggested  that  the  required  frequency  of 
training  to  maintain  proficiency  in  these  tasks  probably  would  have  been  much 
greater  had  the  TM  not  been  used. 

151  Shields,  J.L. ,  Goldberg,  S.L.,  £  Dressel,  J.D.  (1979)  .  Retention  of 
basic  soldiering  skills  (ARI  Research  Report  1225) .  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Field  Artillery  Training  Center  evaluators  (Es)  tested  523  soldiers' 
performance  on  20  basic  common  tasks.  Ss  were  either  in  the  process  of 
completing  entry-level  training  or  had  completed  training  during  the  previous 
12  months.  Es  rated  task  performance  "GO"  or  "NO  GO"  for  each  task  step  and 
for  the  task  as  a  whole.  Tasks  varied  in  the  rate  at  which  percent  of  "GO" 
declined  since  training.  Three  factors  accounted  for  most  of  the  differences 
in  retention:  number  of  task  steps,  order  of  original  training,  and  the 
presence  or  absence  of  subtasks.  Task  steps  that  tended  to  be  forgotten  were 
those  that  were  not  suggested  by  the  previous  sequence  of  steps  or  by  the 
equipment.  It  is  suggested  that  findings  can  be  used  to  maintain  desired 
levels  of  proficiency  in  critical  skills  and  to  determine  which  tasks  require 
frequent  training. 

054  Gade,  P.A.,  Fields,  A.F.,  Maisano,  R.E.,  £  Marshall,  C.F.  (1980). 

Training  approaches  and  data  entry  methods  for  semi-automated  command  and 
control  systems .  Proceedings  of  tha  Human  Factors  Society-2 4th  Annual 

Meeting- 

In  Experiment  I,  a  response-sensitive  instructional  strategy  was  conpared  to 
more  traditional  instructional  strategies  in  an  embedded  training  program 
designed  for  manual  data  entry  operators  in  the  Army  TOS  command  and  control 
system.  Results  showed  that  using  the  response-sensitive  strategy  reduced 
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training  tine  without  reducing  inputting  accuracy.  Experiment  2  examined  the 
relative  efficiency  of  four  different  manual  data  entry  methods:  Typing, 
Typing  with  an  error  corrector.  Menus,  and  Auto-completion  with  an  English 
option.  Results  show  that  Menus  are  the  most  accurate  inputting  method.  No 
differences  were  found  in  entry  time  among  the  methods.  Results  also  showed 
little  agreement  between  preference  and  performance  of  the  methods. 

120  Morrison,  J.E.,  t  Goldberg,  S.L.  (1982).  A  cognitive  analysis  of  armor 
procedural  taak  training  (ARI  Technical  Report  €05)  .  Alexandria,  VA: 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Compares  traditional  and  performance-oriented  approaches  to  procedural 
training  and  notes  their  deficiencies.  A  cognitive  interpretation  of 
procedural  learning  is  advanced,  and  training  implications  are  discussed. 
Representative  armor  procedures  are  analyzed  to  derive  the  underlying  memory 
structures  required  for  recall.  Specific  training  applications  of  the  memory 
structures  that  might  help  the  soldier  learn  and  remember  a  procedure  (e.g., 
"part -task*  training;  association  of  subgoal  names  with  subprocedures)  are 
discussed. 

144  Schendel,  J.D.,  &  Hagman,  J.D.  (1982) .  On  sustaining  procedural  skills 
over  a  prolonged  retention  interval.  Journal  of  Applied  Psychology.  £1(5), 
605-610. 

Examined  the  relative  costs  and  benefits  of  3  alternative  approaches  for 
sustaining  procedural  skills  over  a  prolonged  retention  interval  and  whether 
individuals  can  estimate  in  advance  of  retention  testing  how  much  refresher 
training  they  require  to  regain  proficiency.  38  Army  reservists  were  assigned 
to  groups  according  to  their  past  experience  on  the  experimental  task- 
disassembly  /assembly  of  a  machine  gun.  One  group  was  trained  to  criterion  and 
then  received  100%  initial  overtraining.  This  group's  8-week  retention  and 
retraining  performance  was  compared  against  that  of  a  group  that  received  the 
same  amount  of  additional  training  midway  through  the  retention  interval.  The 
control  group  trained  to  criterion  but  received  no  additional  training  prior 
to  retention  testing  and  retraining.  Refresher  training  estimates  were 
collected  immediately  prior  to  retention  testing.  Cost  effectiveness 
considerations  weighed  heavily  in  favor  of  the  overtraining  group.  Ss 
appeared  to  know  how  much  refresher  training  they  required  to  regain 
proficiency. 

036  Dossett,  D.L.,  &  Hulvershorn,  P.  (1983).  Increasing  technical  training 
efficiency:  Peer  training  via  computer-assisted  instruction.  Journal  of 
Applied  Psychology,  M(4),  552-558. 


Conducted  two  studies  of  peer  training  via  computer-assisted  instruction  (CAI) 
in  the  technical  training  of  electronics  in  the  military.  Ss  were  182  male 
U.S.  Air  Force  personnel.  Study  1  compared  a  peer-training  CAI  group  (n  -  72) 
to  both  an  individually  trained  CAI  group  (n  -  55)  and  a  conventionally 
trained  group  (n  -  55) .  Results  indicate  no  differences  in  achievement 
levels,  but  the  mean  training  times  of  both  CAI  groups  were  significantly  less 
than  that  of  the  conventionally  trained  group.  The  mean  peer-trained  CAI 
training  time  was  significantly  lower  than  that  of  the  individually  trained 
group,  and  the  variability  of  training  time  was  also  significantly  lower. 

Exp.  II,  using  the  peer-trained  subsample  from  Exp  I  and  an  additional  48  Ss, 
investigated  optimum  pairing  strategies.  Ss  were  paired  for  training  based  on 
their  previously  demonstrated  ability.  Pairing  strategies  did  not  affect 


individual  achievement  scores,  but  some  strategies  were  consistently  superior 
in  reducing  training  times.  Implications  of  CAI  peer  training  in  both 
military  and  civilian  technical  training  are  discussed. 

066  Hagman,  J.D.,  &  Rose,  A.M.  (1983).  Retention  of  military  tasks:  A 
review.  Human  Factors .  25/(2),  199-213. 

Discusses  13  experiments  conducted  or  sponsored  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  These  experiments  examined 
the  retention  of  military  tasks  performed  in  an  operational  environment  and 
provide  empirical  data  on  the  effects  of  method,  task,  and  ability  variables 
that  influence  attempts  to  improve  retention  through  training.  These  data 
provide  information  useful  to  trainers  and  training-program  developers  in 
deciding  how,  what,  and  whom  to  train  to  enhance  retention  and  to  achieve 
greater  training  management  effectiveness. 

092  Johnson,  W.B.,  £  Fath,  J.L.  (1983).  Design  and  initial  evaluation  of 
mixed-fidelity  courseware  for  maintenance  training.  Proceedings  of  the 
Human  Factors  Soclety-27th  Annual  Meeting.  1017-1021. 

Results  are  reported  of  a  project  which  involved  the  design  of  computer-based 
courseware  for  maintenance  training.  The  courseware  includes  a  training 
simulation  which  combines  two  levels  of  fidelity.  The  microprocessor-based 
courseware  was  developed  for  troubleshooting  training  on  an  electronic 
telephone  switchboard  at  the  U.S.  Army  Signal  School  in  Fort  Gordon,  Georgia. 
Initial  evaluation  of  the  courseware  demonstrated  that  computer-based 
simulation  can  be  effectively  used  to  supplement  or  partially  replace  "hands- 
on"  practice  with  real  equipment.  Practice  on  the  simulation,  interspersed 
with  real  equipment  practice,  added  a  new  and  valuable  dimension  to  the 
diagnostic  training. 

101  Knerr,  C.M.,  Keller,  S.D.,  &  Laurence,  J.H.  (1984).  Training  strategies 
for  the  Ml  Ah ram*  tank  driver  training  (ARI  Research  Report  1383) . 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Classified  Ml  Abrams  tank  driver-trainer  (DT)  tasks  according  to  the  training 
effectiveness  and  cost  effectiveness  prediction  model  that  prescribes  learning 
guidelines  based  cn  behavioral  activities,  conditions,  standards,  and  feedback 
of  the  tasks.  Most  of  the  tasks  were  procedural,  but  many  procedures  required 
voice  communications,  decision  making,  or  both.  One  set  of  the  DT  program 
presented  the  continuous  movements  tasks  of  driving  such  as  steering.  Some 
learning  guidelines  were  common  to  all  DT  tasks  (e.g.,  providing  active 
practice  and  feedback),  while  others  were  specific  to  the  type  of  task  (e.g., 
high  fidelity,  continuous  feedback  for  continuous  movement  tasks) .  Potential 
DT  features  pertained  to  all  tasks  (e.g.,  scoring  tasks)  or  to  specific  tasks 
(e.g.,  increasing  the  number  and  repetition  of  decision-making  tasks). 
Integration  of  the  DT  into  the  program  of  instruction  considers  use  of  the  Ml 
tank  technical  manual,  new  programs  orienting  the  trainee  to  the  driving  block 
of  instruction  and  the  driver's  intercom,  and  changes  in  the  device  hardware 
and  software. 
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180  Wrightman,  D.C.,  &  Lintem,  G.  (1985).  Part-task  training  for  tracking 
and  manual  control.  "»■»«"  22,(3)/  267-283. 

Reviews  the  effectiveness  of  part-task  training  for  low-aptitude  or 
inexperienced  students  and  establishes  a  conceptual  structure  to  guide 
research.  Issues  pertaining  to  tracking  in  manual  control  are  emphasized. 
Transfer  of  training  is  emphasized,  and  crucial  features  of  the  methodology 
and  of  means  of  assessing  transfer  are  also  discussed.  The  part -task 
procedures  of  segmentation,  fractionation,  and  simplification  are  explained, 
and  procedures  for  reintegrating  parts  into  the  whole  task  are  summarized. 
Implications  for  video-gaming  skills  are  noted. 

024  Ciaverelli,  A.P.  (1986).  Instructional  strategies  for  improving  training 
system  effectiveness.  Proceedings  of  the  Human  Factors  Society-  30th 
Annual  Meeting.  144-148. 

Advanced  simulators  and  other  training  systems  have  been  designed  to  replicate 
as  realistically  as  possible  various  operational  systems  and  environments. 

The  rationale  for  this  particular  design  goal  is  to  attain  sufficient  fidelity 
for  maximum  training  transfer  to  the  real  world.  Yet,  effective  training  may 
not  only  depend  on  attaining  adequate  fidelity,  but  also  upon  how  a  given 
training  system  is  used  to  achieve  specific  learning  outcomes.  This  paper 
identifies  some  of  the  salient  problems  with  training  system  design  and  use, 
and  discusses  possible  solutions  for  improving  their  training  value  based  on 
incorporating  instructional  support  capabilities  and  using  prescriptive 
instructional  methods. 

087  Hunt,  J.P.,  &  Evans,  K.L.  (1986).  The  development  and  evaluation  of  a 
moving  target  engagement  training  program  with  the  M16A1  rifle. 

Proceedings  of  the  Human  Factors  Society  -  3Qth  Annual  Meeting,  285-289. 

This  proposal  reports  the  findings  of  one  of  a  series  of  eight  experiments 
concerned  with  improving  the  soldier's  ability  to  engage  moving  personnel 
targets.  This  overall  research  effort  examined  the  effect  of  different  modes 
of  fire,  additional  ammunition,  methods  of  target  engagement,  the  use  of  an 
intermediate  live-fire  exercise,  and  the  use  of  device-based  training  on 
moving  target  engagement.  Current  (J.S.  Army  training  for  Infantrymen  includes 
one  day  of  instruction  in  moving  target  engagement.  This  experiment  describes 
a  modifies  training  program  with  special  training  devices  conducted  over  a 
two-day  period. 

137  Roman,  J.H.,  Pistone,  R.A.,  &  Stoddard,  M.L.  (1986).  A  trilevel 

interactive  design  model  for  pilot  part-task  training.  Proceedings  of  the 
Human  Factors  Society  -  30th  Annual  Meeting.  126-130. 

Development  of  effective,  scenario-driven  training  exercises  requires  both  an 
instructional  design  and  a  delivery  system  that  match  the  subject  domain  and 
needs  of  the  students.  The  Training  Research  Team  at  Los  Alamos  National 
Laboratory  conducts  research  and  development  of  prototype  training  systems. 

One  of  the  Team's  efforts  is  a  joint  research  project,  supported  with  funding 
and  behavioral  science  guidance  from  the  Army  Research  Institute,  to  develop  a 
prototype  part -task  trainer  for  student  helicopter  pilots.  The  Team  designed 
a  "trilevel  interaction"  model  and  a  Level  III  interactive  video  disk  delivery 
system  for  this  project.  The  model,  founded  on  instructional  and  psychological 
theory,  should  be  transferable  to  other  domains  where  part -task  training  is 
appropriate . 
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116  Mayer,  S.J.,  6  Russell,  J.S.  (1987) .  Behavior  modeling  training  in 

organizations:  Concerns  and  conclusions.  Journal  of  Management-.  12(1), 
21-40. 

Reviews  studies  of  behavior  modeling  (BM) ,  a  popular  training  methodology  in 
organizational  settings.  It  is  concluded  that  the  reasons  for  BMs 
effectiveness  are  not  as  clear  as  supposed  and  that  there  are  4  issues  that 
remain  unresolved:  (1)  Internal  and  external  threats  to  validity  from  field 
research  are  minimal,  but  variations  in  training  programs  do  not  rule  out 
threats  to  the  construct  validity  of  behavioral  and  performance  improvements 
in  the  work  place.  (2)  There  is  no  evidence  that  BM  is  more  cost  effective 
than  other  training  methods.  (3)  A  variety  of  BM  techniques  is  used  in  the 
training  programs,  and  little  study  has  been  completed  on  which  techniques  are 
the  most  effective.  (4)  No  studies  have  examined  the  long-term  shifts  in  the 
values  and  attitudes  of  trainees.  Future  research  should  focus  on  these 
concerns  in  evaluating  the  utility  of  BM  methodology  in  organizational 
training  and  development  interventions. 
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Strategics  -  Additional  Titles 

Maintenance  training  and  sinulation:  Design  processes  and 

evaluation  criteria .  7  (015) 

A  technique  for  choosing  cost-effective  instructional  delivery 

systems .  11  (012) 

Learning  guidelines  and  algorithms  for  types  of  training  objectives.  13  (001) 

Efficacy  of  a  training  priorities  model  in  an  Army  environment .  14  (057) 
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Cm«  and  Taut  Training 

075  Harris,  J.H.,  Osborn,  W.C.,  t  Boldovici,  J.A.  (1977).  Reserve  component 
training  for  operating  and  maintaining  the  M4BA5  tank  (ARI  Technical 
Report  77-A14) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

Identifies  constraints  to  consider  in  planning  National  Guard  tank  training 
and  presents  an  outline  for  a  crew-level  program  for  the  M46A5  tank,  using 
existing  training  materials  as  much  as  possible.  The  Crew  Interaction 
Performance  Test  and  Duty  Position  Readiness  Tests  were  developed  to  measure 
individuals'  proficiency  in  a  crew  environment  and  their  ability  to  perform  4 
crew  tasks,  respectively. 

152  Shriver,  E.L.,  Jones,  D.R.,  Hannaman,  D.I».,  Griffin,  &  Sulzen,  R.H. 

(1979)  .  BMMlfflB—nfc  nf  anail  CQabRt  arm”  unit-  leader  tactical  training 
techniques  and  a  model  training  system  (ARI  Research  Report  1219) . 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

This  research  developed  simulation  techniques  for  small-unit  leaders  and 
tactical  decision  making  while  minimizing  troop  participation.  Basic 
instructional  principles  underlying  engagement  simulation  were  used  to 
construct  a  map  board  game  abstraction  of  field  exercises.  The  game  was  a 
two-sided,  free-play  map  exercise  for  teaching  infantry  tactics  to  platoon 
leaders.  As  originally  conceived,  junior  officers  could  play  the  game  to 
learn  tactical  skills  which  they  would  later  apply  during  REALTRAIN  exercises 
of  troops. 

However,  for  best  results  leaders  appeared  to  need  to  learn  to  work  with  their 
noncommissioned  officers  (NCOs)  and  then  to  practice  in  the  field,  both  with 
and  without  troops.  A  variation  of  the  basic  game  enabled  the  unit  team  of 
platoon  and  squad  leaders  to  work  together  on  the  game  board  as  they  would 
during  an  actual  engagement. 

Concurrently,  a  field  training  technique,  focused  on  leader/subordinate 
interaction,  was  explored.  The  field  opposition  exercise  involved  only  key 
leaders  on  each  side,  without  troop  support. 

A  combined  arms  map  board  game  was  also  developed  to  help  familiarize  junior 
officers  with  the  nature  of  tactical  operations  that  integrate  mechanized 
infantry,  armor,  and  anti-armor  elements  against  a  deployed  enemy. 

The  research  summarized  in  this  report  has  produced  (a)  an  infantry  squad/ 
platoon  level  map  board  game  for  two-player  or  multiplayer  use;  (b)  an 
infantry  squad/platoon  level  field  opposition  exercise;  and  (c)  a  combined 
arms  platoon/company  level  map  board  game.  Separate  documents  are  available 
on  each  of  the  three  training  techniques. 

046  Finley,  D.L.,  £  Dyer,  J.L.  (1980).  Army  teams:  Who  are  they  and  how  do 
they  function?  Proceedings  of  the  Human  Factors  Society  -  24th  Annual 

Meeting. 

The  Army  Research  Institute  Field  Unit  at  Fort  Banning  has  initiated  a  long 
range  research  program  in  the  areas  of  team  training  and  evaluation.  The 
focus  is  on  small  (2-11  persons),  formally  recognized.  Army  work  teams  in  the 
combat  and  combat  support  branches.  The  overall  research  game  plan  is  to 
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concentrate  on  identifying:  (1)  What  it  is  that  constitutes  team,  as  opposed 
to  individual,  performance;  (2)  Which  of  these  team  performance  skills  can  be 
modified  by  training  and  also  serve  to  determine  team  output;  (3)  Analytic  and 
assessment  techniques  for  identifying  and  evaluating  these  skills;  and  (4) 

Team  training  techniques  appropriate  for  Army  conditions.  The  anticipated 
final  research  products  include  Instructional  Systems  Development  (ISD) 
techniques  and  guidance  regarding  team  training  program  development,  team 
performance  assessment  techniques,  and  improvement  and/or  development  of 
selected  team  training  and  assessment  programs. 

The  purpose  of  this  presentation  is  to  report  on  two  steps  that  have  been 
accomplished  in  this  program  thus  far.  The  first  of  these  is  the  development 
of  an  inventory  of  Army  teams.  The  second  is  a  preliminary  identification  of 
what  constitutes  team  performance  at  the  functions  level. 

033  Denson,  R.W.  (1981) .  Team  Training;  Literature  review  and  annotated 

bibliography  (Technical  Report  No.  80-40).  U.S.  Air  Force  Human  Resources 
Dab,  Logistics  &  Technical  Training  Div.,  Wright -Patterson  Air  Force  Base, 
OH:  U.S.  Air  Force  Human  Resources  Lab. 

Reviews  government  documents,  industry  reports,  journal  articles,  and  books  on 
team  training,  to  determine  what  is  currently  known  and  to  identify  unresolved 
issues  relevant  to  team  training  by  the  U.S.  Air  Force.  Documents  dated 
before  1960  and  social-psychological  studies  of  small  group  behavior  are 
limited  to  a  few  review  articles.  Topics  covered  by  the  documents  reviewed 
are  definitions;  individual,  task,  and  team  characteristics;  organization  and 
structure  of  teams;  feedback;  objectives,  measurement,  and  evaluation  of  team 
training;  and  the  development  of  instructional  systems  for  effective  training. 
The  appendix  contains  an  annotated  list  of  129  items. 

034  Denson,  R.W.  (1981) .  Team  training:  Literature  review  and  annotated 
bibliography.  Catalog  of  Selected  Document a  In  Psychology,  11,  92. 

086  Hritz,  R.J.,  Roth,  J.T.,  Lewis,  C.M.,  Finley,  D.L.,  Dyer,  J.L.,  & 

Strasel,  H.C.  (1983).  A  generic  model  for  evaluating  team  performance. 
Proceedings  of  the  Human  Factors  Society  -  27th  Annual  Meeting.  1048-1052. 

The  purpose  of  this  paper  is  to  discuss  a  model  or  methodology  for  evaluating 
the  proficiency  or  performance  of  military  teams.  The  methodology  is 
appropriate  for  evaluating  the  performance  of  any  team  type  performing  any  of 
its  assigned  mission.  The  methodology  contains  procedures  for  assessing  both 
team  process  (team  behavior)  and  team  outcome  (team  success) .  Application  of 
the  methodology  to  a  specific  team  performing  a  specific  mission  generates 
information  which  is  useful  in  identifying  team  training  deficiencies  and  in 
establishing  team  training  requirements.  The  procedure  is  also  useful  for 
assessing  the  effects  of  team  training,  i.e.,  for  measuring  the  amount  and 
kind  of  team  proficiency  realized  as  a  result  of  specific  training. 

168  Swezy,  R.W. ,  £  Salas,  E.  (1987).  Development  of  instructional  design 
guidelines  for  team  training  devices.  Proceedings  of  the  Human  Factors 

Society  -  31at  Annual -Meeting,  97-ioi. 

Both  the  research  data  base  and  practical  experience,  indicate  that  engineers 
involved  in  the  design  and  development  of  team  training  devices  either  have 
inadequate  access  to,  or  for  other  reasons  do  not  typically  use,  human  factors 
and  instructional  design  principles.  This  paper  discusses  a  project  whose 
purpose  is  to:  (1)  identify  team  processes  and  characteristics  which  may  be 
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employed  in  the  design  of  team  training  devices,  (2)  state  this  information  in 
guideline  form,  and  (3)  begin  the  development  of  a  taxonomy  of  team  training 
design  characteristics  which  may  be  used  to  organize  the  guidelines . 


138  Stout,  R.J.,  Cannon-Bowers,  J.,  Morgan  B.B.,  6  Salas,  E.  (1989).  The 
development  of  a  scale  to  assess  the  teamwork  needs  of  training 
situations.  Proceedings  of  Human  Factors  Society  -  33rd  Annual 

Meeting.  1268-1272. 

Operational  studies  have  revealed  a  need  to  focus  attention  on  team  training, 
and  a  need  for  effective  teamwork  skills  for  successful  training  performance. 
The  present  study  was  designed  to  develop  an  assessment  scale  that  can  be  used 
by  instructors  of  various  training  situations,  which  will  yield  a  measure  of 
the  degree  of  teamwork  required  in  their  situations.  Data  obtained  from  the 
scale  show  psychometrically  sound  properties  of  the  scale  (high  internal 
consistency  and  high  item-total  correlations)  and  initial  validity  of  it  (the 
ability  to  distinguish  various  training  situations  as  to  the  extent  of 
teamwork  that  is  required) .  Recommendations  for  future  research  are  also 
discussed. 


Crew  and  Team  Training  -  Additional  Titles 

Behavioral  data  in  the  design  of  aircrew  training  devices .  12  (030) 
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iMdcr  and  Instructor  Training 

175  Wallins,  R.S.,  Rumsey,  M.G.,  6  Gilbert,  A.C.  (1980).  Analysis  of  junior 
Officer  training  needs  (ARI  Research  Report  1236).  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Results  from  interviews  and  surveys  indicate  that  most  problems  described  by 
U.S.  Army  junior  officers  and  corroborated  by  noncommissioned  officers  and 
enlistees  were  of  an  interpersonal  and  organizational  nature  (e.g., 
relationships  with  subordinates,  military  justice,  discipline) .  The  training 
and  courses  that  cadets  received  in  these  areas  were  rated  as  extremely 
valuable.  Data  suggest  several  ways  to  improve  precommissioning  training, 
including  a  greater  interaction  with  active  Army  personnel;  more  on-the-job 
experience  before  commissioning;  the  use  of  problem-oriented,  realistic 
training;  and  more  emphasis  on  interpersonal,  "soft-skill''  training. 

035  DiPaolo,  A.J.,  6  Patterson,  A.C.  (1983).  Selecting  a  training  program 
for  new  trainers.  Training  and  Development  Journal.  22(1),  96-101. 

To  assist  training  directors  in  selecting  the  most  appropriate  programs  for 
their  personnel,  the  basic  competencies  for  the  inexperienced  trainer  were 
identified  and  37  "train  the  trainer*  packages  were  analyzed.  Many  weaknesses 
in  these  training  programs  were  noted,  including  a  failure  to  match  program 
content  to  the  organizational  setting,  a  lack  of  participant  self-assessment 
and  diagnostic  procedures,  and  the  omission  of  a  formal  program  follow-up 
evaluation  and  behavior  maintenance  conponent .  The  specific  skills  that  new 
instructors  should  gain  from  training  programs  include  an  understanding  of 
adult  learning  and  motivational  techniques,  communication,  and  correct  use  of 
media  and  resources  during  presentation. 
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PREDICTION  AND  EVALUATION  OF  STUDENT  AND  MEDIA  PERFORMANCE  EFFECTIVENESS 


Prediction 

176  Wheaton,  G.R.,  Mirabella,  A.,  £  Farina,  A.  (1971).  Trainee  and 

instructor  taak  quantif icat.ion : _ Development  of  quantitative  indic&a  and  a 

predictive  methodology  (NAVTRAEQUIPCEN  No.  69-C-0278-1) .  Orlando,  FL: 

Naval  Training  Equipment  Center. 

An  exploratory  study  was  undertaken,  as  part  01:  a  program  to  develop 
quantitative  techniques  for  prescribing  the  design  and  use  of  training 
systems.  As  a  first  step  in  this  program,  the  present  study  attempted  to  (a) 
compile  an  initial  set  of  quantitative  indices,  (b)  determine  whether  these 
indices  could  be  used  to  describe  a  sample  of  trainee  tasks  and  differentiate 
among  them,  (c)  develop  a  predictive  methodology  based  upon  the  indices,  and 
(d)  assess  that  methodology  using  studies  in  the  literature.  The  compilation 
included  the  display-evaluative  index,  a  set  of  panel  layout  indices,  and  a 
set  of  task  rating  scales.  These  indices  were  applied  to  task  analytic  data, 
collected  on  sonar  operator  trainers.  Application  of  the  indices  proved 
feasible,  and  differentiation  among  3  training  devices,  and  within  4  trainee 
subtasks  (set-up,  detection,  localization,  classification)  was  possible.  The 
predictive  method  which  was  generated  was  an  adaptation  of  the  standard 
multiple  regression  model.  Mean  task  scores  replaced  the  usual  individual 
criterion  scores,  and  quantitative  task  index  values  were  used  as  predictor 
scores.  This  adaptation  was  tested  using  data  from  published  studies  on 
tracking.  Significant  multiple  correlations  using  task  indices  were  found  for 
criterion  data  obtained  during  early  stages  of  practice.  A  combination  of 
task  and  training  indices  did  predict  later  performance.  This  result 
supported  the  contention  that  a  prescriptive  method  must  include  "training"  as 
well  as  "task  indices"  in  order  to  account  for  advanced  levels  of  proficiency. 

124  Neal,  G.L.,  Dannhaus,  D.M.,  Tierney,  T.J.,  &  Cartner,  J.A.  (1977).  First 
may  or  may  not  be  best,  or  cost  and  training  effectiveness  analysis  in 
comparing  training  approaches.  Proceedings  of  the  Human  Factors  Society  - 

21st  Annual  Meeting. 

In  the  future  the  human  factors  specialist  can  expect  to  plan  and  conduct 
economic  analysis-oriented  training  studies  called  cost  and  training 
effectiveness  analysis  (CTEA) .  This  paper  enumerates  basic  CTEA  principles 
and  describes  a  basic  rifle  marksmanship  training  study  to  illustrate  their 
application. 

100  Klein,  G.C.,  Kane  J.J.,  Chinn,  A. A.,  &  Jukes,  A.O.  (1978).  Analyzing 
training  device  effectiveness  in  cases  where  test  data  is  inconclusive. 
Proceedings  of  the  Human  Factors  Society  -  22nd  Annual  Meeting.  152-156. 

Budgetary  and  time  constraints  continue  to  turn  the  trainer  toward  the  use  of 
simulative  devices  in  designing  training  programs.  Sophisticated  devices 
engineered  to  simulate  often  conqplex  man-machine  interfaces  usually  are  major 
investments  for  the  Government  Agency  or  Corporation  who  use  them.  How  can 
the  user  ensure  that  the  device  is,  in  fact,  worth  the  investment?  The 
obvious  answer  to  the  question  is  to  conduct  prototype  testing  using 
representative  students  to  measure  the  degree  of  Training  Transfer.  Testing 
requires  a  further  substantial  investment,  however,  in  terms  of  both  time  and 
money.  In  many  cases,  proper  test  data  is  just  not  available.  Can  device 
effectiveness  be  measured  when  marginal  test  data  or  no  test  data  at  all 
exists?  The  authors,  through  four  successive  Cost  and  Training  Effectiveness 


Analyses  commissioned  by  the  United  States  Army,  have  attempted  to  make 
training  effectiveness  assessments  in  just  such  cases .  Through  the 
application  of  a  behavioral  predictive  model  known  as  TRAINVICE ' (Wheaton, 
1976),  they  have  been  moderately  successful  in  determining  device  utility  in 
terms  of  potential  training  transfer.  This  paper  specifically  discusses  the 
most  recent  study,  that  of  the  Ground/Vehicular  Laser  Locator  Designator  and 
its  associated  prototype  trainer.  The  paper  describes  the  test  data  available 
at  the  beginning  of  the  study,  discusses  the  strengths  and  weaknesses  of  the 
TRAINVICE  Model  as  applied  in  the  study,  and  contributes  some  thoughts 
pertaining  to  the  future  directions  that  studies  of  this  nature  might  take. 

022  Carter,  G.,  &  Trollip,  S.  (1979).  A  constrained  maximization  extension 
to  incremental  transfer  effectiveness,  or.  How  to  mix  your  training 
technologies .  Proceedings  of  the  Human  Factors  Society  -  23rd  Annual 

Meeting. 

The  incremental  transfer  effectiveness  concept  can  be  extended  to  a 
constrained  maximization  context  by  replotting  the  data  to  form  a  set  of  iso¬ 
transfer  contours  and  adding  constraints  to  this  same  graph  as  binding  linear 
functions.  This  transformation  permits  an  analytically  elegant  solution  to 
maximizing  transfer  of  training  dynamically  constrained  by  financial, 
environmental,  reliability,  or  other  restricting  factors  for  any  number  of 
instructional  technologies  through  application  of  the  Lagrange  Multiplier 
method.  Alternatively,  the  dual  problem  of  finding  the  minimum  value  of  these 
constraints  can  be  solved  to  achieve  a  specified  transfer  of  training.  An 
example  of  applying  the  Lagrange  Multiplier  method  to  training  is  provided  and 
the  potential  usefulness  of  mathematical  programming  techniques  noted.  This 
extension  is  particularly  useful  in  complex  training  environments  involving 
numerous  instructional  technologies  or  frequently  changing  constraints.  In 
the  single  instructional  technology  case,  this  extension  provides  additional 
graphical  insight  into  the  trade-offs  involved  in  designing  cost-effective 
training  programs. 

Ill  Maitland,  A.J.  (1980).  Using  predictor  equations  to  assess  potential 

performance  of  armor  enlistees  (ARI  Research  Report  1240)  .  Alexandria,  VA: 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Examines  research  findings  on  predicting  the  performance  of  U.S.  Army  armor 
crewmen  and  shows  how  these  findings  may  be  applied  in  the  current  training 
program.  Results  of  various  methods  of  assigning  groups  of  enlistees  to 
training  programs  illustrate  the  effects  of  different  decision  rules  on  the 
end-product  of  training.  With  the  method  of  prediction  supplied  in  this 
review,  the  performance  potential  of  groups  of  enlistees  may  be  assessed  from 
readily  available  test  scores,  and  the  result  of  assigning  these  individuals 
to  different  training  programs  may  be  estimated. 

115Matlick,  R.K.,  Berger,  D.C.,  Knerr,  C.M.,  &  Chiorini,  J.R.  (1980).  Cost 
and  training  effectiveness  analysis  in  the  Army  life  cycle  systems 
management  modal  (ARI:  Litton  Mellonics  TR  503).  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

This  report  examines  the  requirements  and  purposes  of  cost  and  training 
effectiveness  analysis  (CTEA)  in  the  context  of  the  Army's  Life  Cycle  Systems 
Management  Model.  Current  CTEA  methodologies  are  examined.  Methods  to 
augment  existing  models  are  developed  and  a  general  synthesized  model  for  use 
by  CTEA  analysts  is  proposed. 
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032  Dawdy,  E.D.,  &  Hawley,  J.K.  (1982).  A  forecasting  method  for  training 
effectiveness  analysis.  Proceedings  of  the  Human  Factors  Society  -  26th 
Annual  Meeting.  250-254. 

In  dealing  with  emerging  materiel  systems  early  in  their  life  cycle,  the 
selection  of  an  acceptable  training  program  often  requires  the  use  of  a 
training  effectiveness  forecast  methodology.  Methods  traditionally  used  to 
determine  training  effectiveness  require  data  that  are  not  available  until 
after  many  important  training-related  decisions  have  been  made.  A  training 
effectiveness  forecasting  method  is  described  that  combines  the  development  of 
viable  training  program  alternatives,  the  estimation  of  individual  task 
training  effectiveness,  the  estimation  of  training  program  effectiveness,  and 
the  integration  of  cost  data  to  determine  the  most  cost-effective  training 
approach.  The  methodology  has  the  advantage  of  fixing  neither  effectiveness 
nor  cost.  Rather,  the  procedure  allows  for  later  consideration  of  either 
constraint  as  part  of  the  selection  criterion.  Although  the  emphasis  of  the 
presentation  is  on  the  estimation  of  training  effectiveness,  extended  uses  of 
the  resulting  data  are  noted,  including  assessing  trainability,  training  time 
requirements,  and  requirements  for  maintenance  or  refresher  training. 

125  Neal,  G.L.  (1982).  Overview  of  training  effectiveness  analysis. 

Proceedings  of  the  Human  Factors  Society  -  26th  Annual  Meeting.  244-248. 

U.S.  Army  Training  and  Doctrine  command  (TRADOC)  training  effectiveness 
analysis  (TEA)  began  in  1975  and  has  evolved  into  today's  TRADOC  TEA  System. 

An  overview  of  the  TEA  System  history,  characteristics,  methodology,  scope, 
and  lessons-leamed  are  presented  in  this  paper. 

172  Tufano,  d.r.,  &  Evans,  R.a.  (1982).  The .  prediction,  of  .-training  device 

fif fectiyeneaa ; _ A  review  of  Army  models  (ARI  Technical  Report  613) . 

Alexandria,  VA:  U.S.  Amy  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

This  report  is  a  critical  review  of  the  analytic  models  which  have  been 
developed  by  the  Army  to  predict  training  device  effectiveness.  In 
particular,  special  consideration  is  given  to  a  family  of  models  known 
collectively  as  TRAINVICE.  Although  they  are  all  intended  to  provide  an  index 
of  transfer  potential,  these  four  models  differ  in  several  important  ways. 

The  variables  considered  in  the  calculation  of  the  indices  are  given  different 
degrees  of  emphasis  (or  weight)  in  each  model.  In  fact,  in  some  cases  a 
variable  is  ignored  completely,  e.g.,  the  instructional  features  of  a  training 
device.  The  preparatory  analyses  employed  to  estimate  the  values  of  each 
variable  change  a  great  deal  from  model  to  model.  Moreover,  the  combinatorial 
procedures  used  to  calculate  an  index  of  effectiveness  from  the  variable 
values  are  very  different  in  each  mod  goal  of  this  report  is  to  present  the 
principal  points  of  difference  among  the  various  predictive  models,  and  thus 
facilitate  a  comparative  assessment  of  their  merit.  The  results  of  the 
analysis  and  subsequent  validation  will  provide  a  basis  for  the  selection/ 
refinement  of  one  or  more  of  the  models  for  incorporation  into  a  defined  set 
of  procedures  for  specifying  training  device  effectiveness. 
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102  Knerr,  C.M.,  Nadler,  L.B.,  Dowell,  S.K.,  i  Tufano,  D.R.  (1983). 

Comparison  of  training  transfer  and  ef fectivenesc  models.  Proceedings  of 
»he  Human  Factors  Society  -  27th  Annual  Meeting.  906-910. 

The  costs  of  training  devices  and  simulators  have  induced  the  military  to 
formulate  models  for  predicting  the  training  effectiveness,  including  training 
transfer  effectiveness,  of  devices  during  their  design  and  development . 
Analysis  of  existing  models  compared  them  on  the  following  dimensions: 
objectives,  components,  units  of  analysis,  metrics,  and  development. 
Development  included  level  of  completion,  validation,  and  automation.  The 
models  analyzed  were  those  for  predicting  effectiveness  or  prescribing  device 
characteristics  rather  than  military  Instructional  Systems  Development  (ISD) 
model  and  the  Navy's  Training  Effectiveness  and  Cost  Effectiveness  Prediction 
model.  Other  models  were  the  Training  Device  Effectiveness  model  (TRAINVICE) ; 
Training  Efficiency  Estimation  Model;  training  resource  estimators;  manpower, 
personnel,  and  training  estimation  models;  multi-attribute  utility  estimation; 
and  methods  for  specifying  training  device  features.  Most  of  the  models  were 
prescriptive,  rather  than  predictive,  of  effectiveness.  Few  (including 
TRAINVICE)  estimated  transfer  to  the  operational  setting. 

103  Knerr,  C.M.,  Nadler,  L.B.,  &  Dowell,  S.K.  (1984).  Training  transfer  and 
effectiveness  models  (AIR:  HumRRO  Final  Report).  Alexandria,  VA:  U.S. 

Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Several  models  have  been  developed  to  predict  training  effectiveness  or 
training  transfer  to  performance  on  operational  equipment.  One  model, 
TRAINVICE,  is  undergoing  validation  and  improvement .  This  report  compares 
TRAINVICE  and  other  models  to  identify  techniques  and  components  to  augment 
those  in  TRAINVICE. 

Review  of  the  literature  and  contacts  with  cognizant  scientists  identified  the 
following  models  for  comparison: 

Training  Effectiveness  and  Cost  Effectiveness  Prediction  (TECEP) 

Instructional  Systems  Development  (ISD) 

Training  Device  Specification  Models 
Training  Device  Design  Guide 
Taxonomy  of  Training  Device  Features 
How  to  Determine  Training  Device  Requirements  and 
Characteristics 

Simulator  Training  Requirements  and  Effectiveness  Study  (STRES) 

Cost  and  Effectiveness  Analysis  of  Training  Programs 
Training  Efficiency  Estimation  Model  (TEEM) 

Training  Consonance  Analysis  (TCA) 

Training  Developer' s  Decision  Aid  (TDDA) 

Training  Development  Decision  Support  System  (TDDSS) 

Manpower,  Personnel,  and  Training  (MPT)  Estimation  in  the  Systems 
Acquisition  Process 

Cost  and  Training  Effectiveness  Analyses  (CTEA)  in  Army 
Material  (sic)  Acquisition 

Military  Manpower  and  Hardware  Procurement  (HARDMAN) 

Method  of  Designing  Instructional  Alternatives  (MODIA) 

Training  Requirements  Analysis  Model  (TRAMOD) 
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Coordinated  Human  Resources  Technology  (CHRT) 

Acquisition  of  Supportable  System  Evaluation  Technology  (ASSET) 

The  analysis  compared  the  models  on  five  dimensions:  Objectives,  units  of 
analysis,  components,  metrics,  and  level  of  development. 

Objectives  and  units  of  analysis  divide  the  models  into  two  groups:  those 
pertaining  to  features  within  training  devices  and  those  pertaining  to  whole 
training  programs.  Predictive  models  tend  to  have  more  quantitative  measures 
and  metrics  than  prescriptive  ones.  Models  that  rely  on  subjective  judgments 
may  better  distinguish  among  alternative  designs  if  they  use  comparative 
judgments  (rather  than  the  composites  of  absolute  judgments  currently  used) . 
Most  of  the  models  have  not  been  validated,  and  cannot  be  unless  measures  of 
criteria  can  be  developed  and  applied. 

Future  research  on  predictive  and  prescriptive  models  of  training 
effectiveness  and  transfer  can  use  the  comparisons  for  potential  model 
components,  methods,  and  applications. 

109  Levinson,  E.D.,  £  Donovan,  M.  (1984).  Simulator  fidelity  specification 
based  on  training  needs.  Proceedings  of  the  Human  Factors  Society  -  28th 
Annual  Meeting.  142-146. 

This  report  summarizes  the  work  to  date  on  a  simulator  qualification 
methodology  project.  The  development  of  a  simulator  qualification  process, 
based  on  a  training  perspective,  is  described  so  that  it  can  be  used  on  other 
qualification  projects.  The  measures  used  to  quantify  the  simulator's 
performance  are  provided  and  the  results  of  demonstration  analysis  are 
described. 

128  Pitts,  E.N.,  Salas,  E.,  Terranova,  M.,  Allen,  G.L.,  &  Morgan,  B.B. (1984) . 
Dynamic  measures  of  potential  training  success.  Proceedings  of  the  Human 
FactQta,  Sflciety.z.  28th  Annual  Meeting,  855-859. 

The  Armed  Forces  and  other  large  organizations  often  use  scores  from 
standardized  aptitude  batteries  as  indicators  of  cognitive  aptitude.  However, 
aptitude  may  also  be  demonstrated  by  the  learning  that  occurs  during  training 
and  measured  by  parameters  such  as  initial  ability  levels  and  time  needed  to 
acquire  information  or  skills.  By  using  computer  administered  Complex 
Experimental  Learning  Tasks  (CELTS),  learning  rate  parameters  recently  have 
proved  to  be  pragmatic  as  well  as  theoretical  indicators  of  final  performance. 
Potential  advantages  of  this  approach  include  ease  and  economy  of  computer 
administration,  testee  acceptance  of  job  relevant  tests,  and  potential 
benefits  of  shortened  training  schedules.  The  current  research  compared  rate 
measures  derived  from  learning  on  four  CELTs  with  a  paper-and-pencil  battery 
designed  to  include  static  aptitude  measures  of  the  same  domains.  Performance 
measures  were  computed  from  stimulus  display  times,  subject  response  time,  and 
item  accuracy.  Overall  final  performance  was  computed  using  the  average  of 
the  last  five  minutes.  Correlational  analyses  and  regression  indicate  that, 
with  some  qualifications,  learning  rate  measures  are  predictors  of  individual 
and  overall  levels  of  performance  on  each  CELT.  Implications  of  these 
findings  are  that  the  current  practice  of  using  static  aptitude  tests  for 
selection  to  training  programs  may  not  provide  the  most  accurate  picture  of  an 
individual's  potential  success  or  failure  in  that  program,  and  that,  given  the 
trend  towards  new  computer-assisted  training  technologies,  individuals  may  be 
selected  on  the  basis  of  their  potential  for  rapid  learning,  thus  making  use 
of  the  least  expensive  and  most  efficient  training  methods  possible. 
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126  Neal,  G.L.,  £  Paris,  T.L.  (1985).  The  contribution  of  training 

effectiveness  analysis  in  the  Army.  Proceedings  of  the  Human  Factors 
Society  -  29th  Annual  Meeting. 

The  U.S.  Army  Training  and  Doctrine  Command's  (TRADOC)  Training  Effectiveness 
Analysis  (TEA)  System  studies  assume  that  measured  soldier  job  performance  is 
a  function  of  the  interaction  of  five  factors:  the  soldier,  the  trainer,  the 
training  subsystem,  the  hardware  subsystem,  and  the  training  environment. 
Examples  of  how  this  assumption  guides  TEA  studies  and  contributes  to  the 
analysis  of  Army  training  effectiveness  are  provided. 

138  Rose,  A.M.,  Wheaton,  G.R.,  £  Yates,  L.G.  (1985).  Forecasting  device 

effectiveness :  T .  Issues  (ARI  Technical  Report  680) .  Alexandria,  VA: 

U.S.  Army  Institute  for  the  Behavioral  and  Social  Sciences. 

Discusses  issues  that  bear  on  the  development  of  formal  analytic  methods  for 
predicting  the  potential  effectiveness  of  alternative  training  devices. 
Theoretical  issues  discussed  include  the  meaning  of  the  term  "device 
effectiveness,"  how  transfer  of  training  is  measured,  its  use  as  a  measure  of 
device  effectiveness,  and  alternatives  to  transfer  of  training  as  measures  of 
effectiveness . 

139  Rose,  A.M.,  Wheaton,  G.R.,  £  Yates,  L.G.  (1985).  Forecasting  device 
effectiveness:  II.  procedures  (ARI  Research  Report  85-25) .  Alexandria, 

VA:  U.S.  Army  Institute  for  the  Behavioral  and  Social  Sciences. 

Describes  an  analytic  training  device  effectiveness  forecasting  technique 
(DEFT)  that  accounts  for  device  effectiveness  in  terms  of  several  different 
criteria  and  classes  of  independent  predictor  variables.  In  its  present  form, 
DEFT  is  a  series  of  interactive,  menu-driven  computer  programs  that  provide 
for  three  levels  of  device  evaluation  that  are  chosen  as  a  function  of  the 
type  of  input  data  that  are  available  and  the  degree  of  diagnosticity  that  is 
needed.  A  DEFT  user's  manual  and  listings  of  DEFT  programs  are  appended. 

140  Rose,  A.M.,  Martin,  A.W.,  Yates,  L.G.  (1985).  Forecasting  Device 
Effectiveness  (Volume  31  (ARI  Technical  Report  681).  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

Several  analytic  procedures  were  conducted  to  address  various  aspects  of  the 
scalar  properties  of  the  device  effectiveness  forecasting  technique  (DEFT) . 
these  procedures  included  Monte  Carlo  simulations  to  assess  the  interpretation 
of  DEFT  output,  sensitivity  of  DEFT  parameters,  comparison  of  outputs, 
stability,  and  inter-rater  agreement.  Results  indicated  that  it  would  be 
necessary  to  incorporate  assumptions  regarding  expected  distributions  of  input 
variables  in  order  to  meaningfully  interpret  DEFT  output.  Also,  the  Monte 
Carlo  analyses  demonstrated  the  sensitivity  of  DEFT  output  scores  to 
variations  in  inputs,  and  assessed  the  effects  of  various  assumptions 
regarding  measurement  error  on  output  scores .  The  inter-rater  agreement  issue 
was  addressed  by  having  several  raters  apply  DEFT  to  the  three  actual  training 
devices.  Results  indicated  a  high  degree  of  consistency  among  raters  for  all 
devices  for  all  levels  of  DEFT. 
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145  Schendel,  J.D.,  Heller,  F.H.,  Finley,  D.L.,  t  Hawley,  J.K.  (1985).  Use 
of  Weaponeer  marksmanship  trainer  in  predicting  M16A1  rifle  qualification 
performance,  unman  EacfcflXfl,  22,(3),  313-325. 

In  Exp  I,  69  initial  entry  soldiers  were  assigned  to  3  groups  varying  in 
Weaponeer  target  scenario  difficulty  and  were  tested  twice  on  a  scenario  prior 
to  firing  record  fire  (RF) .  Weaponeer  performance  under  the  most  difficult 
scenario  appeared  to  be  the  best  predictor  of  RF  performance.  Predictions 
improved  when  later  shots  and  firing  positions  were  considered  on  the  device. 
In  Exp  II,  2,244  permanent  party  troops  were  divided  into  5  groups  according 
to  amount  and  type  of  prior  training  on  Weaponeer.  Results  indicate  that 
weaponeer  can  be  used  to  predict  RF  performance  as  long  as  training  is  not 
provided  immediately  prior  to  Weaponeer  testing. 

090  John,  P.G.,  Klein,  G.A.,  t  Taynor,  J.  (1986).  Comparison-based 

prediction  for  front-end  analysis.  Proceedings  of  the  Human  Factors 
Society-3 0th  Annual  Meeting.  149-153. 

Comparison-Based  Prediction  is  a  formalized  method  for  using  analogical 
reasoning  to  generate  estimates  early  in  conceptual  design  and  planning.  Its 
feasibility  has  been  demonstrated  for  structuring  expert  judgment  in  a  variety 
of  domains,  including  human  factors  applications  such  as  predicting  the 
effectiveness  of  training  devices. 

002  Adams,  A.V.,  &  Rayhawk,  M.  (1987).  A  review  of  models  of  cost  and 

training  effectiveness  analysis  (CTEA).  Volume  TT ; _ Cost  analysis  (ARI 

Research  Note  87-59).  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

Cost  and  training  effectiveness  must  be  considered  jointly  in  considering  how 
to  meet  training  requirements.  Cost  predictions  for  various  training 
alternatives  are  not  at  their  most  valuable  in  the  early  stages  of  a  weapon 
system's  life-cycle,  and  this  report  discusses  the  problems  encountered  with 
data  availability  at  this  stage.  A  cost  model  must  incorporate  economic 
concepts  as  opposed  to  purely  accounting  costs.  The  economic  ideas  embedded 
in  CTEA  are  discussed,  and  the  search  for,  and  difficulties  in  developing  and 
applying,  a  generic  CTEA  model  are  addressed.  Finally,  building  on  earlier 
reviews,  17  cost  models  and  guidebooks  are  examined  in  terms  of  purpose  of 
development,  features  included,  usefulness  in  the  early  stages  of  weapon 
system  development,  and  strengths  and  weaknesses  of  the  model  or  guidebook. 

In  a  summary  on  the  lessons  to  be  learned,  the  more  advanced  state  of  training 
effectiveness  development  is  noted;  suggestions  are  made  for  new  research  on 
costing;  continued  early  coordination  between  system  designer,  training 
developer,  and  cost  analyst  is  advocated;  and  comparison-based  methods  are 
recommended  as  one  means  to  address  data  problems  in  the  early  stages  of 
system  development . 

018  Cantor,  J.A.  (1988) .  Research  and  development  into  a  comprehensive  media 
selection  model.  Journal  of  Instructional  Psychology.  15.(3),  118-131. 

Describes  an  instructional  systems  media  selection  model  based  on  training 
effectiveness  and  cost  effectiveness  prediction  techniques  developed  to 
support  the  U.S.  Navy's  training  programs.  The  nrocess  is  a  2-stage  model  that 
addresses  both  training  and  cost  effectiveness  considerations  in  selecting  an 
instructional  delivery  system. 
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060  Goldberg,  I.  (1988).  Training  of factiveness  and  coat  iterative  technique 
(tacit)  volume  I:  Training  effectiveness  analysis  (ARI  Research  Note  88- 
35) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

This  research  note  describes  the  effectiveness  model  of  the  Training 
Effectiveness  and  Cost  Iterative  Technique  (TECIT) ,  a  new  model  concerned  with 
the  cost  effectiveness  of  training  devices  and  simulators  (TD/S)  at  all  phases 
of  the  life  cycle  development.  Volume  II  describes  the  cost  model. 

For  the  purposes  of  this  study,  the  effectiveness  of  a  training  device  or 
simulator  was  defined  as  a  function  of  the  following  factors:  safety, 
acquisition  learning  on  the  TD/S,  transfer  of  training  from  the  TD/S  to  an 
exercise  on  the  weapon  system  during  training,  job  or  battle  readiness,  and 
the  utilization  ratio  of  the  TD/S. 

A  research  strategy  is  outlined  in  the  research  note.  This  strategy  considers 
cross-sectional  and  longitudinal  designs,  TD/S  life  cycle  phases,  and  various 
validity  designs  (e.g.  discriminant,  concurrent,  and  predictive  validity) . 
Sampling  of  subject  matter  experts'  opinions  and  TD/S  is  also  considered. 

Also  included  in  this  document  is  a  review  of  related  models,  including  the 
Device  Effectiveness  Forecasting  Technique  (DEFT) ,  Forecasting  Training 
Effectiveness  (FORTE),  and  Comparison  Based  Prediction  (CBP) .  A  comparison  of 
model  features  is  also  included,  along  with  sample  questionnaires,  and  an 
illustrative  data  base. 

003  Adams,  a.v,  t  Rayhawk,  m.  (1988) .  Training  effectiveness  and  coat 

iterative  technique  (TECTT)  Volume  TT  : Coat  effectiveness  analysis  (ARI 

Research  Note  88-57) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

This  research  note  discusses  an  important  issue,  the  training  of  soldiers  in 
the  use  of  a  weapon  system,  which  is  important  to  the  operational  cost  and 
success  of  the  system  in  the  carrying  out  of  its  mission.  Cost  and  Training 
Effectiveness  Analysis  (CTEA)  models  formalize  the  process  of  choosing  among 
alternative  training  technologies  by  comparing  the  cost  and  training 
effectiveness  of  these  technologies.  In  this  report,  an  economic  framework  is 
presented  for  integrating  cost  and  training  effectiveness  data  for  CTEA 
studies.  The  framework  builds  on  transfer  effectiveness  models  and  offers  a 
model  for  the  effectiveness  of  cost-minimizing  training  technologies.  The 
model  is  particularly  well-suited  to  the  selection  of  training  technologies 
for  tasks  that  are  trained  on  weapons  systems  whose  operation  is  costly  or 
life  threatening.  The  research  note  shows  how  cost  data  would  be  developed 
for  the  model  in  evaluating  computer-assisted  instruction  as  a  training 
technology  for  the  Ml  Abrams  tank. 

027  Companion,  M.A.,  Carson,  R.,  Garris,  R.  (1990).  ASTAR  Operational 
Evaluation  Report  (NAVTRASYSCEN  Special  Report  90-005) .  Orlando,  FL: 

Naval  Training  Systems  Center. 

The  Automated  Simulator  Test  and  Assessment  Routine  (ASTAR)  is  an  automated 
decision  aid  designed  to  assist  a  training  system  analyst  in  the  prediction  of 
the  effectiveness  of  training  devices.  ASTAR  was  developed  to  provide  a 
systematic  and  analytic  evaluation  procedure  to  aid  training  device  design  and 
acquisition.  Prior  to  implementation  as  a  standard  evaluation  technique  ,  it 
was  necessary  to  conduct  field  tests  with  operational  analysts  to  determine 
user  acceptance  of  ASTAR.  The  objective  was  to  compare  and  contrast  ASTAR  to 
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other  automated  Device  Effectiveness  Technologies  (DETs)  and,  if  warranted,  to 
formulate  a  plan  to  implement  ASTAR  as  a  standard  evaluation  technique  within 
the  DoD  Instructional  System  Development  (ISD)  process.  The  operational 
evaluation  was  accomplished  through  three  operational  tests  and  a  longitudinal 
test  using  operational  training  systems  and  their  analysts.  The  tests 
examined  performance,  utility,  and  user  issues  with  regard  to  ASTAR,  and  Media 
Selection  (AIMS)  System,  while  the  concept  of  ASTAR  was  well  received  by  the 
operational  analysts,  the  current  iiqplementation  of  ASTAR  achieved  poor  user 
acceptance.  ASTAR  requires  extensive  enhancement  before  it  can  gain  general 
approval.  A  Functional  Description  (ASTAR  Operational  Evaluation: 

Conclusions  and  Recommendations,  Companion,  1991)  for  an  improved  ASTAR  (ASTAR 
XI)  was  developed  which  addressed  the  problems  of  ASTAR.  It  is  recommended 
that  any  further  developmental  action  on  ASTAR  be  limited  to  consideration  of 
ASTAR  II. 

028  Companion,  M. A.  (1991).  ASTAR  Operational  Evaluation:  Conclusions  and 
Rarnimendationa  ( NAVTRAS Y S CEN  Special  Report  90-004) .  Orlando,  FL:  Naval 
Training  systems  Center. 


The  Automated  Simulator  Test  and  Assessment  Routine  (ASTAR)  is  an  automated 
decision  aid  developed  to  provide  a  systematic  evaluation  procedure  to  aid 
training  device  design  and  acquisition.  ASTAR  was  “field  tested:  by  training 
analysts  to  provide  joint  services  operational  evaluation  of  the  device  prior 
to  its  being  widely  distributed.  The  results  of  the  operational  tests 
indicated  that  the  current  ASTAR  program  is  unacceptable  for  widespread  use, 
although  the  concept  of  a  decision  aid  for  training  analysts  was  met  with 
enthusiasm.  The  problems  with  ASTAR  were  primarily  with  the  user  interface. 
The  problems  with  ASTAR  were  primarily  with  the  user  interface.  The  program 
is  cumbersome  and  tedious  to  use,  and  the  output  is  hard  to  interpret.  There 
is  also  a  problem  with  the  current  con piler  which  precludes  smooth  operation 
on  the  80286  (like  the  Zenith  248)  class  computers.  It  is  recommended  that 
the  ASTAR  software  be  improved  (as  described  in  the  Functional  Description 
contained  in  the  appendix  of  this  document)  before  any  further  distribution 
of  the  program  takes  place. 


Prediction  -  Additional  Titles 

Optimisation  of  simulation-based  training  systems:  Model 

description,  implementation,  and  evaluation .  21  (164) 
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069  Hall,  E.R.,  Rankin,  W.C.,  &  Aagard,  J.A.  (1976).  Training  effectiveness 
aaaaaaneaLl  IX  Problems,  concepts.  and  evaluation  alternatives  (TAEG 
Report  No.  39) .  Orlando,  FL:  Training  Analysis  and  Evaluation  Group. 

Examines  specific  problems  affecting  U.S.  Navy  training  evaluation  programs 
and  discusses  technical  considerations  relevant  to  evaluation  and  training 
effectiveness  assessment.  General  procedures  for  assessing  the  effectiveness 
of  Navy  training  courses  are  given,  and  methodological  options  for  evaluation- 
data  gathering  are  described  and  evaluated.  Recommendations  are  made  for 
improving  training  evaluation  practice  and  for  establishing  a  Training 
Effectiveness  Assessment  Center  to  assist  in  the  planning  and  conduct  of  Navy 
training  evaluations. 

110  Maher,  P.D.  (1976-77) .  Evaluation  of  learning,  performance,  and 

productivity.  Journal  of  Educational  Technology  Systems.  1(4),  341-353. 

Presents  a  seminar  paper  which  explored  some  of  the  requirements  for 
understanding  evaluation  goals  and  techniques  in  designing  learning  systems. 
The  author  attempted  to  clarify  what  is  meant  by  the  term  "applied  learning 
technologist."  The  concept  is  approached  in  layman's  terms,  concentrating  on 
the  need  for  validating  both  the  "learning"  requirements  and  the  corresponding 
"learning  system"  design.  Suggested  contents  of  a  curriculum  for  an  "applied 
learning  technologist"  certificate  program  are  discussed.  The  results  of  a 
survey  questionnaire  are  summarized. 

104  Knerr,  C.M.,  Downey,  R.G.,  &  Kessler,  J.J.  (1977).  Training  individuals 
in  Army  units:  Comparative  effectiveness  of  selected  training  extension 
course  lessons  and  conventional  methods.  Catalog  of  Selected  Documents  in 
Psychology,  2(20). 

V 

107  Laabs,  G.J.,  &  Panell,  R.C.  (1977).  Development  of  job-based  diagnostic 
test  in  an  individualized  instruction  program  for  boiler  technicians. 
Proceedings  of  the  Human  Factors  Society  -  21st  Annual  Meeting. 

A  criterion-referenced  test  keyed  to  an  individual,  self-paced  instruction 
program  developed  as  part  of  a  diagnostic  testing/shipboard  training  system. 
The  job-based  test  described  hypothetical  situations  that  were  based  on  known 
job  requirements.  Under  each  such  situation,  questions  were  asked  that 
required  the  demonstration  of  skills  and  knowledges  that  supported  the  job  and 
were  covered  in  the  various  modules  of  the  instruction  program.  High  face  and 
content  validity  were  ensured  by  using  cards,  charts,  diagrams,  and 
illustrations  in  presenting  each  job  situation;  and  by  having  job  experts 
write  test  items.  Items  were  selected  that  best  discriminated  between 
graduates  and  nongraduates  of  a  shore-based  program  using  the  same 
instruction,  the  cutoff  score  for  the  set  of  items  pertaining  to  each  module 
was  determined  from  the  performance  of  graduates  of  the  shore-based  program 
and  applied  to  a  cross-validation  sample.  Test  scores  from  two  administrations 
were  used  to  estimate  the  reliability  of  the  diagnostic  decision  making. 
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023  Carter,  R.  J.  <1982) .  Methodologies  for  evaluating  training  products  and 
processes .  Proceedings  of  the  Human  Factors  Society  -  26th  Annual 
Meeting.  258-262. 

Methodologies  for  conducting  training  effectiveness  evaluations  (TEE)  of 
ongoing  Amy  training  were  detailed.  The  design  process,  consisting  of  a 
literature  search  and  formative  evaluations,  was  discussed.  The  developed  TEE 
system  was  described. 

112 Maitland,  A.J.  (1982).  Training  effectiveness  analysis:  where  the 

operator  meets  the  equipment.  Proceedings  of  the  Human  Factors  Society  - 
26th  Annual  Meeting.  255-257. 

A  Training  Effectiveness  Analysis  (TEA)  was  performed  during  the  preparation 
for  the  final  operation  testing  of  the  Ml  (ABRAMS)  Main  Battle  Tank.  The 
results  of  the  analysis  of  the  training  program  for  tank  operators  are 
discussed  to  illustrate  the  usefulness  of  TEA  in  the  developmental  process  of 
Army  training  programs. 

161  Sorenson,  P.H.,  6  Pennell,  R.  (1982).  Technical  training: _ development 

of  instruction  treatment  alternatives  (U.S.  AFHRL  TR  82-32).  Brooks  Air 
Force  Base,  TX:  Air  Force  Human  Resources  Laboratory. 

Provides  guidelines  for  the  development  and  evaluation  of  alternative 
instructional  approaches  that  hold  promise  of  improving  instructional 
effectiveness,  with  emphasis  on  how  to  identify  and  test  interactive 
relationships  between  individual  differences  among  learners  and  instructional 
conditions  or  treatment.  Problems  of  measurement  that  are  basic  to  diagnosis 
and  evaluation  are  discussed. 

179  Witmer,  B.G.,  &  Kristiansen,  D.M.  (1982).  The  development  and  field 
trial  of  a  system  fox  evaluating  the  effectiveness  and  efficiency  of  a 
training  program  (ARI  Research  Report  1336).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Developed,  tested,  and  refined  a  system  to  assist  training  managers  or 
evaluators  increasing  the  effectiveness  of  training  programs.  The  Training 
Program  Evaluation  (TPE)  system  involves  direct  observation  of  training  and 
testing  rather  than  second-hand  accounts  by  trainers  or  trainees.  The  TPE 
utilizes  tests  routinely  given  after  each  block  of  instruction.  A  major  test 
of  the  system  came  when  it  was  used  to  evaluate  the  effectiveness  of  the  Ml 
tank  transition  training  program  and  to  suggest  program  improvements.  The 
utility  of  the  system  was  demonstrated.  Several  agencies  have  used  the  TPE  to 
gather  training  effectiveness  information  that  was  subsequently  used  to 
improve  the  training  programs.  The  training  observation  form,  a  training 
opinion  questionnaire,  TPE  data  collection  forms  and  lesson  learned  during 
field  trials  of  the  TPE  systems  will  help  military  managers  avoid  costly 
mistakes  in  conducting  their  own  training  program  evaluations. 

177  Wick,  D.T.,  Ruffner,  J.W.,  £  Bickley,  W.R.  (1984).  Conserving 

instructional  training  resources  through  use  of  the  personalized  system  of 
instruction.  Proceedings  of  the  Human  Factors  Society  -  28th  Annual 
Meeting.  375-378. 

Training  managers  in  the  U.S.  Army  face  a  continuing  problem  of  meeting 
increasing  training  demands  with  limited  instructional  resources,  particularly 
qualified  training  personnel.  In  most  aviation  unit  flight  training  programs. 
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instructor  pilot  (IPs)  serve  a  dual  a  role  of  academic  lecturer  and  in-the- 
cockpit  instructor.  To  reduce  the  amount  of  IP  time  required  to  administer 
training  in  a  reserve  aviator  training  program,  a  19-day  training  program  was 
developed  that  incorporates  home  study  and  features  of  the  Personalized  System 
of  Instruction  (PSI)  .  Forty-seven  aviators  were  trained  using  this  program. 
Following  one  year  of  no  training,  24  of  the  aviators  returned  for  refresher 
training  using  the  same  program.  The  results  indicate  that  the  home  study/PSI 
approach  reduces  on-site  academic  training  time  and  virtually  eliminates  IP 
time  on  academic  topics.  The  results  also  suggest  that  an  academic  training 
program  based  on  PSI  can  be  used  successfully  while  conserving  the  scarce 
instruction  resource  of  IP  time. 

064  Graham,  S.E.,  Schlechter,  T.M.,  £  Goldberg,  S.L.  (1986).  a  preliminary 
evaluation  of  model  maintenance  training  program  for  reserve  component 
unit «  (ARI  Research  Report  1421).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

16  U.S.  Army  soldiers  participated  in  either  a  computer-based  maintenance 
simulation  training  for  Ml  turret  and  hull  mechanics  or  a  no-training  control 
procedure.  Ss  who  had  received  the  simulated  troubleshooting  training  made 
fewer  errors  per  period  of  time  on  the  hands-on  transfer  task  than  did 
controls.  The  skills  and  knowledge  developed  in  the  courseware  also 
generalized  to  a  troubleshooting  task  not  specifically  trained. 

005  Andrews,  D.H.,  6  Mohs,  B.  (1987) .  Key  army  decision  maker  concerns  about 
training  performance  measurement  and  assessment.  Proceedings  of  the  Human 
Factors  Society  -  31st  Annual  Meeting.  1251-1255. 

This  study  explored  an  area  of  Army  training  performance  measurement  and 
assessment  (PMA)  which  has  apparently  not  been  examined.  It  provides  an 
understanding  about  Army  training  PMA  requirements  and  uses,  and  reveals  a 
number  of  PMA  issues  which  should  be  more  closely  examined  in  the  future.  The 
methodology  adapted  for  the  study  combined  elements  of  Policy  Capturing 
Analysis  with  elements  of  Policy  Implications  Analysis  and  the  Delphi 
Technique . 

006  Baldwin,  T.T.,  £  Ford,  J.K.  (1988).  Transfer  of  training:  A  review  and 
directions  for  future  research.  Personnel  Psychology.  11.(1),  63-105. 

Critiques  the  research  on  the  transfer  of  training  in  which  learned  behavior 
is  generalized  to  the  job  context  and  maintained  over  a  period  of  time.  The 
effects  of  training  design,  trainee,  and  work  environment  factors  on 
conditions  of  transfer  are  discussed.  Research  gaps  include  the  need  (1)  to 
test  operations  of  training  design  and  work-environment  factors  that  have  an 
impact  on  transfer  and  (2)  to  develop  a  framework  for  conducting  research  on 
the  effects  of  trainee  characteristics  on  transfer.  Needed  advancements  in 
the  conceptualization  and  operationalization  of  the  criterion  of  transfer  are 
also  discussed. 

169  Swezy,  R.W.,  Perez,  R.S.,  £  Allen,  J.A.  (1968).  Effects  of  instructional 
delivery  system  and  training  parameter  manipulations  on  electromechanical 
maintenance  performance.  Human  Factors.  22.(6),  751-762. 

Compared  computer-assisted  instructional  (CAI) -based  delivery  systems  with 
other  training  media  to  determine  the  extent  to  which  instruction  delivered 
via  these  methods  affects  performance  on  electromechanical  maintenance  tasks . 
Opportunity  for  hands-on  practice,  access  to  job  aids,  and  instructional 
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delivery  system  parameters  were  manipulated:  performance  of  110  undergraduates 
(aged  17-43  years)  was  measured  following  training  and  again  after  1  wk. 
Hands-on  practice  of  the  activity  and  access  to  job  aids  during  training  were 
found  to  significantly  improve  performance.  However,  no  differences  in 
performance  of  procedural  maintenance  tasks  were  found  among  Ss  who  were 
trained  using  CAI-based  as  opposed  to  other  lower  technology-based 
instructional  delivery  systems. 
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Evaluation  -  Additional  Titles 


Maintenance  training  and  simulation:  Design  processes  and 

evaluation  criteria .  7  (015) 

An  analytic  process  method  for  defining  appropriate  measures  of 

systems'  performance  and  effectiveness .  7  (045) 

The  con^mter-aic'ed  analytic  process  model:  Operations  handbook 

for  the  analytic  process  model  demonstration  package .  8  (148) 

The  computer-aided  analytic  process  model:  Appendix  to  the 

Operations  Handbook  for  the  APM  demonstration  package .  8  (149) 

On  sustaining  procedural  skill  over  a  prolonged  retention  interval..  25  (144) 

Reserve  component  training  for  operating  and  maintaining  the 

M48A5  tank .  31  (075) 

A  generic  model  for  evaluating  team  performance .  32  (086) 

First  may  or  may  not  be  best,  or  cost  and  training  effectiveness 

analysis  in  comparing  training  approaches .  37  (124) 

Cost  and  training  effectiveness  analysis  in  the  Army  Life  Cycle 

Systems  Management  Model .  38  (115) 

Overview  of  training  effectiveness  analysis .  39  (125) 

The  contribution  of  training  effectiveness  analysis  in  the  Army .  42  (126) 

Training  effectiveness  and  cost  iterative  technique  (TECIT) 

Volume  II:  Cost  effectiveness  analysis .  44  (060) 
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TRAINING  MEDIA 


Features  (Training  Management,  Feedback,  Fidelity) 

159  Smode,  A.F . ,  &  Hall,  E.R.  (1975).  Translating  information  requirements 
into  training  device  fidelity  requirements.  Proceedings  of  the  Human 
Factors  Society  -  19th  Annual  Meeting. 

The  failure  of  training  equipment  to  meet  training  needs  is  frequently 
attributed  to  poor  engineering  fidelity.  Inaccurate  or  inadequate  information 
requirements  may  be  equally  responsible  for  shortcomings  in  device  design. 

Some  observations  and  views  on  fidelity  issues  and  on  achieving  fidelity 
requirements  are  offered. 

042  Eggemeier,  F.T.  (1977).  Two  short-term  techniques  for  gathering  training 
device  requirement  information.  Proceedings  of  the  Human  Factors  Society 
-  21st  Annual  Meeting. 

Two  short-term  questionnaire/interview  techniques  were  used  to  specify  device 
requirements  for  an  aerial  gunnery  part -task  trainer.  The  techniques  differed 
in  the  degree  of  emphases  placed  on  eliciting  expert  pilot  opinion  regarding 
instructional  capability  and  level  of  fidelity  required  in  the  trainer. 
Recommendations  of  pilots  led  to  higher  fidelity  and  more  extensive  simulator 
capability  than  did  similar  estimates  derived  bv  training  and  engineering 
personnel  from  pilots'  descriptions  of  the  task  in  the  operational 
environment.  It  is  concluded  that  both  methods  are  acceptable  short-term 
techniques  for  initial  identification  of  training  requirements.  However, 
long-term  techniques  appear  to  be  optimal  for  making  difficult  decisions. 

047  Finley,  D.L.,  Strasel,  H.C.,  Schendel,  J.D.,  &  Hawley,  <7.K.  (1981). 
Training  device  operationalization  readiness  assessment  capability 
(DORAC) :  The  concept  and  its  implementation .  Proceedings  of  the  Human 
Factors  Society.  -  25th  Annual  Meeting,  416-420. 

Army  operational  forces  are  required  to  report  once  a  month  on  the  readiness 
of  their  units  for  combat.  The  most  difficult  and  subjective  of  the  ratings 
going  into  this  report  is  the  one  concerning  personnel  readiness.  The  problem 
is  one  of  insufficient  and  outdated  data  concerning  the  current  skill  levels 
and  training  deficiencies  of  unit  personnel.  DORAC  may  provide  a  partial 
solution  to  this  problem.  This  paper  presents  the  concept  and  research 
issues;  describes  a  cost-benefits  method  developed  to  provide  training  device 
proponents  and  developers  a  means  to  determine  if  and  how  to  implement  DORAC 
in  the  design  and  utilization  of  a  specific  device;  describes  the  results  of 
applying  the  method  to  exemplary  training  device  design  and  usage  options; 
and,  presents  findings  regarding  the  extent  to  which  one  device  represents  a 
DORAC. 

166  Strasel,  H.C.,  Finley,  D.L.,  &  Hawley,  J.K.  (1982).  Developing  training 
devices  with  an  effective  capability  for  training  management  and  readiness 
assessment.  Proceedings  of  the  Human  Factors  Society  -  26th  Annual 

Meeting,  840-844. 

Army  commanders  and  trainers  need  to  know  the  current  job  performance  levels 
of  their  troops.  They  need  this  information  as  the  basis  for  making  decisions 
about  when  and  where  to  deploy  units  in  battle,  how  much  and  where  to  allocate 
resources  to  training,  what  training  readiness  vaiue  to  enter  in  the  monthly 
Unit  Status  Report,  and  what  specific  training  the  soldiers  should  receive. 
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The  problems  are  that  currently  few  data  are  gathered,  infrequently  (e.g.,  SQT 
data  every  2  years  on  a  decreasing  number  of  skill  areas;  (ARTEPs  rarely!). 

The  reasons  for  these  problems  are  many.  Principal  among  them  are  resource 
constraints  (e.g.,  ammunition,  ranges,  personnel  time)  and  measurement 
capability  limitations  (e.g.,  lack  of  instrumentation  and  inability  to 
directly  observe  in  field/actual  equipment  situations,  and  nonevaluative 
training  devices) .  The  results  are  incomplete  data  that  are  out  of  date,  and 
older  data  are  of  dubious  value  due  to  personnel  turbulence  and  skills  decay. 

029  Connelly,  E.M.  (1984).  Instantaneous  performance  evaluation  with 

feedback  can  improve  training.  Proceedings  of  the  Human  Factors  Society  - 
2flth  Annual  Meeting.  864-868. 

Moment-to-Moment  (MTM)  performance  measures  derived  from  Summary  Performance 
Measures,  provide  a  near  instantaneous  evaluation  of  present  performance 
according  to  how  it  impacts  overall  mission  (i.e.,  summary)  performance.  MTM 
is  a  training  tool  permitting  rapid  feedback  of  performance  information  to  the 
student  as  deviations  from  reference  performance  occur.  While  the  sensitivity 
of  MTM  measures  allow  the  rapid  feedback  at  each  moment  of  the  task, 
experimental  results  suggest  that  such  feedback  should  be  carefully  restricted 
to  enhance  training  efficiency.  Obviously,  the  feedback  must  use  a  mode 
(input  channel)  that  is  non-disrupt ive  to  the  current  task,  but  also  the 
feedback  should  be  modulated  to  focus  on  a  portion  of  the  total  mission 
(presumably  the  part  where  present  performance  most  seriously  degrades  overall 
mission  performance) .  Further  the  trainee  should  be  informed  of  that  portion 
of  the  mission-keyed  feedback. 

This  paper  describes  a  research  program  in  which  MTM  performance  measure  is 
used  to  generate  training  feedback  signals  where  the  feedback  is  keyed  to  the 
mission  resulting  in  the  desirable  performance.  But  the  time  duration  of 
feedback  is  limited  to  a  fixed  percentage  of  the  total  mission  time.  The 
impact  of  MTM  feedback  in  improving  training  efficiency  vs.  feedback  of  only 
the  total  (summary)  task  performance  score  is  presented  along  with  a 
description  of  the  method  used  to  produce  the  MTM  measure  and  the  feedback 
modulation  system. 

171  Thomas,  G.S.,  Kaplan,  I.T.  &  Barber,  H.F.  (1984).  Command  and  control 
training  in  the  combined  arms  tactical  training  simulator  (ARI  Technical 
Report  615) .  Alexandria,  VA;  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

The  Army  Training  Battle  Simulation  System  (ARTBASS)  is  being  developed  to 
succeed  the  Combined  Arms  Tactical  Training  Simulator  (CATTS)  as  the  U.S. 
Army's  most  combined  group  training  system.  The  present  research  assessed  the 
effects  on  command  group  performance  of  the  CATTS/ARTBASS  technology, 
supplemented  by  an  Army  Research  Institute-developed  diagnostic  and  feedback 
package.  Five  battalion  command  groups  participated.  Measures  of  performance 
included  information  flow  during  planning  and  execution,  simulation 
battlefield  outcomes,  and  ratings  of  critical  tasks.  All  performance  measures 
increased  significantly  from  the  first  (pretest)  to  the  fourth  (posttest) 
exercise  day.  In  a  previous  experiment  using  CATTS  without  added  feedback, 
there  was  less  improvement  in  performance.  It  is  concluded  that  CATTS/ARTBASS 
training  is  effective  when  it  includes  detailed  diagnostic  feedback. 
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014  Brett,  B.E.,  Chapman,  W.A.,  £  Hawley,  J.K.  (1986).  Coat  and  information 
af feet iveneaa  analyala:  An  Improved  methodology  (Volume  in  (ARI  Research 
Note  86-05).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

This  report  is  the  second  of  two  volumes  that  present  results  of  an  effort  to 
develop  an  improved  methodology  for  the  conduct  of  Cost  and  Information 
Effectiveness  Analysis  (CIEA) .  CIEA  is  a  methodology  for  the  evaluation  of 
training  device  performance  assessment  capabilities  (D-PACs) .  It  is  directed 
at  the  problem  of  determining  when  the  worth  of  performance  status  information 
available  from  a  D-PAC  offsets  the  costs  required  to  develop,  operate,  and 
maintain  the  capability.  This  report  presents  a  case  study  which  illustrates 
application  of  CIEA  to  evaluate  a  series  of  D-PAC  alternatives  that  provide 
performance  status  information  on  the  gunnery  proficiency  of  crews  for  the 
Combat  Engineer  Vehicle  (CEV) .  CEV  gunnery  D-PAC  alternatives  were  developed 
by  combining  members  of  a  set  of  interactive  video  gunnery  trainers  for  the 
CEV  with  different  types  of  live  fire  exercises .  The  case  study  describes 
activities  conducted  in  the  four  phases  of  the  CIEA  of  CEV  gunnery  D-PACs: 
exploration  of  the  D-PAC  concept  for  CEV  gunnery,  development  of  the  D-PAC 
concept  for  CEV  gunnery,  definition  of  the  CEV  gunnery  D-PAC  alternatives,  and 
evaluation  of  CEV  gunnery  D-PAC  alternatives.  Activities  of  the  last  phase 
culminate  with  specification  of  the  most  cost-effective  CEV  gunnery  D-PAC 
alternative . 

020  Carroll,  R.J.,  &  Dwyer,  D.J.  (1986).  Maintenance  training  simulator 

instructor  station  and  student  station  design  specification.  Proceedings 
of  the  Human  Factors  Society  -  30th  Annual  Meeting.  131-135. 

This  paper  discusses  the  methodology  and  results  of  a  recent  study  conducted 
for  the  Naval  Training  Systems  Center  to  develop  a  generic  Prime  Item 
Development  Specification  for  maintenance  training  simulator  (MTS)  instructor 
and  student  stations  (Carroll,  et  al.,  1986).  The  approach  taken  was  to 
develop  a  classification  scheme  for  MTSs,  develop  a  MTS  instructor 
station/student  station  attribute  taxonomy,  conduct  a  commonality  analysis  of 
instructor/student  station  features  among  the  different  types  of  MTSs,  ajnd 
conduct  a  survey  of  MTSs  to  examine  their  characteristics  and  query 
instructors  on  their  perception  of  the  effectiveness,  of  instructor 
station/student  station  features .  The  survey  results  revealed  that 
instructors  gave  high  effectiveness  ratings  to  most  of  the  existing  instructor 
station  and  student  station  features.  The  homogeneity  of  responses  across  MTS 
types  indicates  that  instructor  station  requirements  are  a  function  of  the 
instructional  requirements  associated  with  providing  maintenance  training  via 
a  training  device,  rather  than  being  determined  by  MTS  physical  or  functional 
characteristics.  The  study  results  were  applied  to  the  development  of  a  model 
specification  for  MTS  instructor  station  and  student  station  design. 

079  Hawley,  J.K.,  Brett,  B.E.,  £  Chapman,  W.A.  (1986).  Cost  and  lnformafion 
effectiveness  analysis:  An  improved  methodology  (Volume  1)  (ARI  Research 
Note  86-04)  .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  Behavioral 
and  Social  Sciences. 

This  report  presents  the  results  of  an  effort  to  develop  an  improved 
methodology  for  the  conduct  of  Cost  and  Information  Effectiveness  Analysis 
(CIEA)  .  CIEA  is  a  methodology  for  the  evaluation  of  training  device 
performance  assessment  capabilities  (D-PACs) .  It  is  directed  at  the  problem 
of  determining  when  the  worth  of  performance  status  information  available  from 
a  D-PAC  offsets  the  costs  required  to  develop,  operate,  and  maintain  the 
capability.  Such  a  determination  may  be  needed  in  order  to  specify 
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requirements  for  a  D-PAC,  or  as  a  basis  for  deciding  between  two  or  more  D-PAC 
design  options  intended  to  satisfy  pre-specified  requirements.  Following  the 
introductory  section,  a  review  of  objective  procedures  for  the  assessment  of 
information  worth  is  presented.  An  objective  method  for  information  worth 
evaluation  based  upon  the  use  of  Combat  Simulation  Models  (CSMs)  is  then 
explored  in  detail.  The  results  of  this  evaluation  based  upon  the  use  of 
Combat  Simulation  Models  (CSMs)  is  then  explored  in  detail.  The  results  of 
this  evaluation  indicated  that  a  CSM-based  CIEA  procedure,  while  technically 
feasible,  is  not  practical.  Section  3  of  the  report  presents  results  from  a 
series  of  formative  tryouts  of  alternative  multi-attribute  utility  measurement 
(MAUM)  procedures  for  conduct  of  CIEA.  Based  upon  these  empirical  results, 
recommendations  for  an  improved  MAUM-based  CIEA  methodology  are  made.  The 
report  continues  with  a  detailed  presentation  of  an  improved  methodology  for 
the  conduct  of  CIEA.  In  this  improved  methodology,  a  series  of  MAUM 
procedures  are  integrated  into  a  standard  cost-effectiveness  framework.  To 
illustrate  the  methodological  description,  an  exemplary  analysis  on  a  set  of 
hypothetical  D-PAC  alternatives  is  included.  Finally,  in  Section  5,  issues 
relevant  to  the  improved  methodology  are  discussed.  Suggestions  for  future 
methodological  development  are  also  presented. 

129  Polzella,  D.J.,  &  Hubbard,  D.C.  (1986).  Utility  and  utilization  of 

aircrew  training  device  advanced  instructional  features.  Proceedings  of 
the  Human  Factors  Society  -  30th  Annual  Meeting.  139-143. 

The  utility  and  utilization  of  the  Advanced  Instructional  Features  (AIFs) 
capabilities  of  USAF  Aircrew  Training  Device  (ATDs)  was  explored  by  means  of  a 
survey  of  534  Simulator  Instructors  from  Air  Training  Command,  Military 
Airlift  Command,  and  Tactical  Air  Command  training  sites.  The  primary  purpose 
of  the  survey  was  to  provide  a  data  base  that  could  be  used  in  defining  the 
requirements  for  ATD  procurements  and  in  developing  future  ATD  training 
programs.  In  general,  the  features  that  were  rated  highest  in  utility  and 
utilization  were  those  used  for  training  management,  variation  of  task 
difficulty/fidelity,  and  monitoring  student  performance.  The  level  of  AIF  use 
was  affected  somewhat  by  hardware  and/or  software  deficiencies;  however,  the 
presumed  training  value  of  an  AIF  was  the  most  important  determiner  of  its 
use. 

056  Gibbons,  S.,  &  Singer,  M.J.  (1989).  Evaluation  of  decision  aids  for 

training  device  design.  Exacesdinga  oL  the  Human  Factors  Society..-^-  33rd 
Annual  Meeting,  1294. 

Newly  developed  cost  and  training  effectiveness  models  are  being  used  by 
training  developers  to  control  costs  and  to  ensure  systematic  training  device 
design.  The  problem  for  the  user  is  how  to  select  the  appropriate  design  aid. 
Unfortunately,  there  are  no  quick  objective  methods  on  which  to  base  this 
selection.  The  selection  decision  for  a  particular  application  can  be  made 
based  on  three  issues.  The  first  issue  is  how  the  design  aid  addresses  device 
instructional  and  fidelity  features.  The  second  issue  is  how  the  design  aid 
formalizes  the  device  design  decision  process.  The  third  issue  is  to  compare 
the  systems  on  their  ease  of  implementation.  Two  decision  aids  are 
analytically  evaluated  on  their  approach  to  training  device  design:  OSBATS 
(Optimization  of  Simulation  Based  Training  Systems),  which  is  in  prototype 
development,  and  ASTAR  (Automated  Simulator  Test  and  Assessment  Routine), 
which  is  ready  to  be  fielded.  These  decision  aids  are  based  on  differing 
theoretical  approaches  to  formalizing  training  device  design.  OSBATS' s 
taxonomy  of  fidelity  features  relates  instructional  features  to  individual 
tasks.  OSBATS  contains  a  trade-off  function  which  uses  historical  cost  and 
benefit  values  for  individual  features.  It  uses  large  amounts  of  detailed 
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information  to  drive  its  algorithms.  ASTAR  is  a  management  tool  which 
organizes  the  diverse  interests  of  a  design  group  to  address  design  issues. 
ASTAR  obtains  judgments  about  instructional  approach  and  device  similarity  for 
each  training  objective.  ASTAR  facilitates  communication  between  members  of  a 
design  team  and  insures  a  consensus  on  the  issues. 

157  Singer,  M.J.,  £  Willis,  R.P.  (1989).  Data  collection  in  support  of  the 
design  of  training  devices.  Proceedings  of  the  Human  Factors  Society  - 
33rd  Annual  Meeting,  1295-1299. 

A  major  problem  in  training  device  design  is  specifying  the  appropriate  level 
of  fidelity  and  required  instructional  features  for  learning.  This  research 
effort  was  designed  to  acquire  detailed  information  about  tasks  and  training 
device  fidelity  features.  The  standard  method  for  developing  information 
about  task  and  fidelity  relationships  has  been  to  conduct  research  into 
training  methods  using  varying  degrees  of  fidelity,  or  to  extrapolate  from 
evaluations  of  training  programs  based  on  newly  developed  training  devices. 

The  rotary-wing  operations  domain  was  selected  as  the  basis  for  gathering 
detailed  relationship  data.  A  Training  Device  Fidelity  analysis  was  conducted 
on  many  of  the  devices  at  the  Army  Aviation  School  at  Ft.  Rucker.  A  survey 
was  then  developed  that  crossed  the  tasks  being  trained  on  the  AH-64  CWEPT 
(Cockpit,  Weapons,  and  Emergency  Procedures  Trainer)  and  the  UH-1  CPT  (Cockpit 
Procedures  Trainer)  with  the  device  characteristics  present  on  those  training 
devices.  The  survey  was  administered  to  instructors  using  the  training 
devices.  The  survey  responses  were  categorized,  and  the  consensus  results  are 
being  used  in  developing  expert  system  rules.  The  conclusion  dram  is  that 
adequate  data  can  be  collected  using  surveys  to  generate  experience-based 
(versus  opinion-based  or  device  evaluation-based)  rules  for  determining 
necessary  and  sufficient  fidelity  aspects  for  training  devices,  where  guidance 
is  inferential  and  opinion-based,  and  where  those  devices  are  costly  and/or 
need  to  be  very  effective. 


57 


TRAINING  MEDIA 


Features  -  Additional  Titles 

Behavioral  data  in  the  design  of  aircrew  training  devices .  12  (030) 

Training  device  operational  readiness  assessment  capability 

(DORAC)  :  feasibility  and  utility .  16  (077) 

Sane  considerations  in  the  design  and  implementation  of  a 

training  device  performance  assessment  capability .  17  (078) 

Designing  training  devices:  The  optimization  of  simulation-based 

training  systems .  20  (156) 

Accommodating  instruction  to  student  characteristics:  Trends 

and  issues .  23  (044) 

Training  approaches  and  data  entry  methods  for  semiautomated 

command  and  control  systems .  24  (054) 

Instructional  strategies  for  improving  training  system 

effectiveness .  27  (024) 

Simulator  fidelity  specification  based  on  training  needs .  41  (109) 

Use  of  Weaponeer  marksmanship  trainer  in  predicting  M16A1  rifle 

qualification  performance .  43  (145) 
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Media  Selection 


131  Reiser,  R.A.,  £  Gagne,  R.M.  (1982).  Characteristics  of  media  selection 
models.  Review  of  Educational  Research.  52,(4),  499-512. 

Identifies  and  evaluates  the  learning  effectiveness  of  the  major  features 
found  in  media  selection  models,  with  emphasis  on  the  characteristics  noted 
across  models.  Features  discussed  include  the  physical  forms  the  models  take, 
the  ways  in  which  they  classify  media,  and  the  media  selection  factors  they 
consider.  Selection  factors  embodied  in  models  affect  media  choices. 
Characteristics  of  learners,  setting,  and  task  are  identified  as  factors  to  be 
given  primary  consideration  in  media  selection. 

106  Kribs,  H.D.,  Simpson,  A.C.,  6  Mark,  L.J.  (1983).  Automated  instructional 
—dll  selection  (ATMS)  (NAVTRAEQUIPCEN  Technical  Report  79-C-0104-1) . 
Orlando,  FL:  Naval  Training  Equipment  Center. 

Developed  and  implemented  the  automated  instructional  media  selection  model  to 
help  reduce  the  cost  and  labor  intensiveness  of  training  program  development 
by  maximixing  the  use  of  pertinent  information  and  automating  the 
non judgmental,  data  manipulation  tasks  of  the  selection  process.  The  user- 
definable  aspect  of  the  system  allows  for  inclusions  of  any  instructional 
characteristics.  It  also  allows  for  better  selection  of  hands-on  training 
devices.  Since  the  system  is  adaptable  to  any  training  situation,  there  is  no 
bias  toward  cognitive,  learning-center  type  media. 
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Madia  Selection  -  Additional  Titles 


Maintenance  training  and  simulation:  Design  processes  and 

evaluation  criteria .  7  (015) 

Implementing  Embedded  Training  (ET) :  Volume  3  of  10 :  The  Role 

of  ET  in  the  Training  System  Concept .  9  (141) 

A  technique  for  choosing  cost-effective  instructional  delivery 

system .  11  (012) 


Research  and  development  into  a  comprehensive  media  selection  model.  43  (018) 
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Training  Devices 


173vestewig,  R.E.,  &  Eggemeier,  F.T.  (1979).  Actual  versus  simulated 
equipment  for  aircraft  maintenance  training:  Cost  implications  of  the 
incremental  versus  the  unique  device.  Proceedings  of  tha  Human  Factors 

Soclftty  -  23rd  Annual  Meeting- 

Life  cycle  cost  estimates  were  developed  for  use  of  simulated  test  equipment 
versus  actual  test  equipment  in  a  maintenance  training  program  of  the  type 
used  for  current  advanced  fighter  aircraft.  Previous  life  cycle  cost 
comparisons  had  not  explicitly  considered  the  cost  implications  of  procurement 
and  support  of  a  unique  training  device  versus  an  incremental  device.  This 
effort  included  the  unique  versus  the  incremental  device  factor.  Total 
estimated  15-year  costs  for  simulated  equipment  trainers  were  significantly 
lower  than  comparable  estimates  for  actual  equipment  trainers.  The  results 
indicate  that  the  cost  implications  of  a  unique  device  versus  an  incremental 
device  are  important  determinants  of  both  acquisition  and  support  cost 
estimates  and  should  be  considered  fully  in  future  life  cycle  costing  efforts. 

021  Carroll,  R. j.  (1983) .  Factors  influencing  the  logistical  support  of 
maintenance  training  simulators:  The  development  of  a  handbook. 
Proceedings  at  the  Human  Factors  Society  -  27th  Annual  Meeting.  1007-1011. 

Experience  with  maintenance  training  simulators  has  revealed  a  number  of 
unanticipated  logistical  support  problems.  In  some  cases,  the  logistical 
support  costs  (i.e.,  the  costs  of  operating,  maintaining,  and  updating) 
incurred  during  the  life  cycle  of  the  simulator  have  been  greater  than 
expected  and  sometimes  have  exceeded  the  initial  acquisition  cost  of  the 
device.  Many  of  these  costs  were  incurred  because  the  personnel  responsible 
for  the  acquisition  of  simulators  were  not  aware  of  the  factors  influencing 
the  life  cycle  costs  or  were  not  aware  of  the  problems  that  could  occur  in 
operating,  maintaining,  and  updating  simulators.  In  addition,  they  had  no 
systematic  procedure  for  evaluating  the  logistical  support  needs  of  the 
maintenance  simulators  they  were  purchasing.  This  report  describes  the 
development  of  a  handbook  that  addresses  these  problems  by  identifying  factors 
which  impact  the  logistical  support  requirements  of  maintenance 
trainers/simulators  throughout  their  life  cycle  and  providing  a  guide  to  help 
predict  and  plan  for  the  logistical  support  required  of  a  given  simulator. 

080  Hays,  R.T.,  &  Singer,  M.J.  (1983).  Research  issues  in  training  device 
design.  Proceedings  of  the  Human  Factors  Society  -  27th  Annual  Meeting. 
147-150. 

The  Army  Research  Institute  (ARI)  has  an  ongoing  research  program  to 
systematically  accumulate  the  data  necessary  to  provide  design  guidance  for 
the  development  of  training  simulators.  As  part  of  this  research  program,  an 
extensive  literature  review  was  conducted.  On  the  basis  of  this  literature 
review,  a  series  of  research  questions,  which  require  empirical  investigation 
was  accumulated.  This  presentation  is  designed  to  bring  these  research  issues 
to  the  largest  possible  audience.  The  goal  of  this  paper  is  to  sensitize 
researchers  to  the  types  of  questions  that  have  the  highest  payoff  for  the 
Army  and  the  training  community.  The  research  questions  are  organized  around 
a  model  of  the  Instructional  Systems  Development  (ISD)  process.  Each  ISD 
section  is  further  subdivided  into  specific  subareas  where  empirical  research 
is  needed. 


Training  Devices  -  Additional  Titles 

How  to  determine  training  device  requirements  and  characteristics: 


A  handbook  for  training  developers .  15  (108) 

Processes  of  skill  performance:  A  foundation  for  the  design 

and  use  of  training  equipment .  16  (162) 

A  demonstration  of  an  expert  system  for  training  device  design .  19  (155) 

A  review  of  models  of  cost  and  training  effectiveness  analysis 

(CTEA)  Volume  I:  Training  effectiveness  analysis .  19  (059) 

Designing  training  devices:  The  optimization  of  simulation-based 

training  systems .  20  (156) 

Instructional  strategies  for  improving  training  system 

effectiveness .  27  (024) 

Comparison  of  training  transfer  and  effectiveness  models .  40  (102) 

Forecasting  device  effectiveness:  I.  Issues .  42  (138) 

Forecasting  device  effectiveness:  II.  Procedures .  42  (139) 

Comparison-based  prediction  for  front  end  analysis .  43  (090) 

Maintenance  training  simulator  instructor  station  and  student 

station  design  specification .  55  (020) 

Evaluation  of  decision  aids  for  training  device  design .  56  (056) 

Data  collection  in  support  of  the  design  of  training  devices .  57  (157) 

Actual  versus  simulated  equipment  for  aircraft  maintenance 
training:  cost  implications  of  the  incremental  versus  the 
unique  device .  61  (173) 

Factors  influencing  the  logistical  support  of  maintenance 

training  simulators:  the  development  of  a  handbook .  61  (021) 
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Coipttt*r-Ba*«d  training  (CBt)  and  Artificial  Intelligence  (AX) 

135  Rigney,  J.N.,  t  Towns,  D.M.  (1970).  TASKTEACH:  A  method  for  computer- 
assisted  performance  training.  Human  Factors.  12/(3),  285-296. 

Describes  a  method  for  using  a  computer  timesharing  system  to  assist  the 
learning  of  serial  tasks.  The  method  is  based  on  mediational  theory.  The 
current  resurgence  of  interest  in  mediational  theory  in  psychological  research 
is  noted,  and  parallels  between  experimental  strategies  and  instructional 
strategies  for  evoking  and  manipulating  mediating  processes  are  pointed  out. 
Categories  of  processes  which  mediate  performance  of  serial  tasks  are 
described.  Procedures  for  facilitating  the  learning  of  these  processes  are 
implemented  by  TASKTEACH.  The  program  sustains  S's  performance  of  complex 
serial  tasks  by  giving  him  variable  amounts  of  support  while  helping  him  learn 
and  organize  the  processes  which  mediate  his  performance  of  these  tasks.  The 
program  generates  output  to  S  during  the  learning  session  by  processing  short 
lists  and  S's  prior  responses.  The  lists  replace  the  conventional  frame-by- 
frame  description  of  an  instructional  sequence  written  in  a  computer-assisted 
instruction. 

009  Blaiwes,  A.S.,  6  Rigney,  J.W.  (1974).  A  guide  for  developing 

performance-structure  oriented  computer-administered  instruction:  theory 
and  research.  Proceedings  of  tha  Human  Factors  Society  -  18th  Annual 
Meeting. 

The  work  reported  here  had  three  main  objectives  with  respect  to  computer- 
administered  instruction  (CAT),  namely,  to  make  CAX:  (1)  easier  to  develop, 

(2)  higher  in  quality,  (3)  more  widely  appreciated  and  accepted.  The  approach 
taken  toward  these  goals  included  two  complementary  kinds  of  efforts.  One 
effort  was  directed  toward  designing  guidelines  and  models  of  CAI  which  can 
assist  training  program  developers  in  their  efforts  to  implement  CAI.  Another 
effort  was  applied  to  the  construction  and  field  evaluation  of  a  trainer  which 
was  based  on  descriptions  and  principles  of  CAX  as  contained  in  the  guidelines 
and  models.  These  efforts  and  some  of  their  results  are  aiscussed  here. 

119  Moore,  M.V.,  Nawrocki,  L.H.,  &  Simutis,  Z.M.  (1979).  The  instructional 
effectiveness  of  three  levels  of  graphics  displays  for  computer-assisted 
Instruction  (TP  359).  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

Ninety  Army  enlisted  personnel  studied  one  of  three  versions  of  a  computer- 
assisted  instruction  (CAI)  lesson  on  the  psychophysiology  of  audition.  The 
text  of  the  lesson  was  identical  for  all  versions;  the  type  of  graphics  used 
in  the  lesson  was  different  for  each  of  the  three  groups.  No  control  group 
was  used.  Graphics  were  either  low  (schematic  representations  and  boxed 
alphanumerics) ,  medium  (line  drawings) ,  or  high  level  (animations  plus  line 
drawings) .  After  completing  the  lesson,  all  Ss  were  tested  for  retention  of 
four  knowledge  categories:  acquisition  and  use  of  principles,  identification 
of  structures,  definitions  and  use  of  terminology,  and  memory  of  specific 
facts.  Results  indicate  that  the  type  of  graphics  used  during  the  CAI  lesson 
did  not  affect  lesson  conpletion  time  or  final  performance  on  the  retention 
tests . 
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097  Kearsley,  G.P.,  k  Hillelsohn,  M.J.  (1982).  Human  factors  considerations 
for  computer-based  training.  Journal  of  Computer-Based  Instruction.  1(4) , 
74-84. 

Reviews  the  literature  on  human  factors  as  it  applies  to  computer-based 
training  (CBT) .  The  relationship  between  human  factors  and  CBT  research  is 
illustrated  by  descriptions  of  studies  dealing  with  six  major  aspects  of  the 
CBT  interface:  the  administration  of  training,  time  management,  instructional 
design/development,  testing  and  development,  student  motivation,  and  hardware. 

088  Jarvis,  M.D.  (1984).  Computer-based  training:  lessons  learned. 

Proo«>edina«  <vf  fha  Human  Factors  Society  -  28th  Annual  Meeting.  515-519. 

Design  and  implementation  of  military  maintenance  training  systems  can  draw 
upon  a  number  of  lessons  learned  from  a  review  of  literature  and  past 
experience  of  applications  of  computer-based  training  technology. 

065  Gray,  W.D.,  Pliske,  D.B.,  &  Psotka,  J.  (1985).  Smart  technology  for 

training; _ Promise  and  current  status  (AR1  Research  Report  1412) . 

Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Discusses  the  Army  Research  Institute' s  Smart  Technology  for  Training  team, 
which  has  monitored  research  in  cognitive  and  computer  science  to  determine 
its  applicability  for  Army  training.  Several  long-term  projects  relate  to  the 
development  and  use  of  mental  models  of  physical  phenomena  and  devices. 
Applications  of  enhanced  computer-assisted  instruction  to  diagnostic  and 
problem-solving  training  are  described. 

098  Kearsley,  G.,  k  Seidel,  R.J.  (1985).  Automation  in  training  and 
education.  Human  Factors,  22(1),  61-74. 

Discusses  the  history  of  automated  instruction,  highlighting  current 
applications,  issues  and  problems,  and  future  prospects.  Current  applications 
include  traditional  uses  of  computers  for  such  tasks  as  testing,  drills, 
tutorials,  games,  simulations,  and  student  management,  while  new  applications 
include  embedded  training,  computer  literacy,  interactive  video  discs,  and 
electronic  lectures.  Currently,  the  most  inport ant  issue  in  automated 
instruction  involves  the  time  and  costs  associated  with  the  development  of 
courseware.  Other  important  problems  are  the  difficulty  of  implementing 
individualized  instruction  in  organizations  accustomed  to  classroom  teaching, 
the  scarcity  of  computer-literate  educators,  and  the  evaluation  of  courseware. 
Microcomputers  have  greatly  accelerated  the  growth  of  computer-based 
instruction  in  all  domains.  Intelligent  computer-assisted  instruction 
programs,  authoring  systems,  hand-held  computers,  speech  processing,  and 
telecommunication  technologies  (e.g.,  video  text)  are  seen  as  shaping  the 
future  direction  of  automated  instruction. 

063  Gott,  S.P.,  Bennett,  W.,  k  Gillet,  A.  (1986).  Models  of  technical 

competence  for  intelligent  tutoring  systems .  Journal  of  Computer-Based 
Instruction,  12(2),  43-46. 

Analyzed  the  problem  solving  performance  of  2  skill  levels  of  apprentice 
electronics  technicians  (N-15)  in  a  complex  military  workplace  to  derive  the 
components  of  technical  competence.  Differences  among  individual  apprentices 
were  derived  through  a  series  of  experiments  that  examined  3  elements  of 
technical  expertise-  procedural  contents  consisting  of  the  goal  structures 
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that  problem  solvers  inpose  on  the  problem  space;  conceptual  knowledge, 
including  explicit  task  structure,  background,  and  contextual  knowledge;  and 
mental  models  that  the  problem  solver  has  available  to  guide  the  initial 
problem  representation.  Results  suggest  that  greater  task  success  by  the 
skilled  performers  was  due  to  superior  mental  models  and  that  conceptual 
knowledge  was  also  a  contributor  to  skilled  performance.  Implications  for 
conplex  skill  training  are  discussed. 

070  Hamel,  C.J.  (1986).  The  CAI  evaluation  checklist:  Guidelines  for  the 
design  of  computer-aided  instruction.  Proceedings  of  the  Human  Factors 
Society  -  30th  Annual  Meeting.  136-138. 

Human  factors  design  guidelines  for  the  development  of  computer-aided 
instruction  are  presented.  The  guidelines  focus  on  the  student-computer 
interaction  and  are  organized  around  five  human  factors  design  principles  for 
computer  systems:  brevity,  consistency,  flexibility,  responsiveness,  and 
compatibility . 

076  Harris,  S.D.,  &  Owens,  J.M.  (1986).  Some  critical  factors  that  limit  the 
effectiveness  of  machine  intelligence  technology  in  military  systems 
applications.  Journal  of  Computer-Based  Instruction.  12(2),  30-34. 

Presents  a  critical  assessment  of  issues  in  military  systems  applications  of 
machine  intelligence  technology.  Areas  discussed  include  relevant  research  in 
artificial  intelligence  and  cognitive  science;  conputat } ~  :ial  theories  of 
thinking;  technological  applications  of  research  in  memory  organization, 
perception,  and  problem  solving;  the  design  of  human-system  interface;  and  the 
augmentation  of  human  performance.  Problems  are  noted  with  respect  to  coping 
with  uncertainty,  vagueness  of  performance  criteria,  and  the  effects  of  stress  * 
on  human-machine  dynamics. 

132  Richardson,  R.J.,  £  Jackson,  T.E.  (1986).  Developing  the  technology  for 
intelligent  maintenance  advisors.  Journal  of  Computer-Based  Instruction. 
13(2),  47-51. 

Describes  a  demonstration  study  to  develop  an  intelligent  maintenance  advisor 
(IMA),  which  is  a  computer-based  device  that  (1)  enables  unskilled  technicians 
to  perform  as  if  they  were  skilled,  (2)  tutors  technicians  who  need  further 
development  in  their  technical  skills,  and  (3)  works  cooperatively  with 
skilled  technicians  to  capture  diagnostic  insights  for  the  benefit  of  peers/ 
successors.  For  the  experienced  technician,  the  IMA  provides  interactive, 
cooperative,  human-computer  problem  solving.  Issues  of  knowledge  representa¬ 
tion  and  end-user  interfaces  are  discussed. 

146  Schlechter,  T.M.  (1986).  An  examination  of  the  research  evidence  for 
computer-based  Instruction  in  military  training  (ARI  Technical  Report 
722) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Reviews  research  on  use  of  computer-based  instruction  (CBI)  in  U.S.  military 
training.  Over  150  papers  and  articles  were  examined,  and  interviews  were 
conducted  with  professionals  in  the  field  of  CBI.  Major  areas  of  focus 
include  costs,  learning  achievements,  individual  differences,  and  student 
attitudes.  It  appears  that  the  evidence  is  inconclusive  with  regard  to 
advantages  of  CBI  over  other  instructional  media  for  reducing  training  time  or 
life  cycle  costs,  facilitating  students'  mastery  of  materials,  accommodating 
individual  learning  differences,  or  motivating  students.  Problematic 
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methodologies  were  found  throughout  the  CBI  literature .  Widespread 
implementation  of  CBI  for  all  educational  situations  is  not  recommended. 
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Confute r-Based  Training  (CBT)  and  Artificial  Intelligence  (AI)  -  Additional 


Titles 

A  guide  for  the  application  of  performance  structure  oriented 

CAI  in  naval  training:  A  working  paper .  11  (134) 

Computer-assisted  instruction:  Decision  handbook .  18  (099) 

Increasing  technical  training  efficiency:  Peer  training  via 

computer-assisted  instruction .  25  (036) 

Design  and  initial  evaluation  of  mixed-fidelity  courseware  for 

maintenance  training . 26  (092) 

A  preliminary  evaluation  of  a  model  maintenance  training  program 

for  reserve  component  units .  49  (064) 

Effects  of  instructional  delivery  system  and  training  parameter 

manipulations  on  electromechanical  maintenance  performance .  49  (169) 
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Other 


031  Davies,  I.K.  (1984).  Fitting  the  media  key  into  instruction.  Training 
and  Development  Journal.  M(12),  22-27. 

Discusses  the  contribution  of  media  to  instructional  design  and  argues  that 
effectiveness  rather  than  efficiency  represents  the  primary  goal  of  media 
usage.  The  design,  development,  and  evaluation  of  instructional  media  require 
skill  in  diagnosis,  decision  making,  and  flexibility.  The  value  of  education 
and  training  in  increasing  employee  reliability  and  improving  performance  is 
considered,  and  the  importance  of  values  in  instructional  technology  is  noted. 
Explicit  and  tacit  dimensions  of  communication  and  knowledge  are  described, 
and  the  marketing  of  the  media  contribution  to  organizational  health  and 
effectiveness  is  discussed. 

048  Finley,  D.L.,  Alderman,  I.N.,  Bogner,  M.S.,  &  Mitchell,  N.B.  (1984). 

Embedded  training  in  the  FOG-M  (Fiber  Optics  Guided  Missile) .  Proceedings 
of  the  Human  Factors  Society  -  2flth  Annual  Meeting.  527-528. 

The  Defense  Science  Board  Summer  Study  of  1982  and  the  Army  Science  Board 
Summer  Study  of  1963  both  recommended  the  incorporation  of  embedded  training 
into  operational  systems  as  one  solution  to  the  shortage  of  unit  resources 
providing  realistic  training  to  operational  Army  personnel.  The  Army  Research 
Institute  has  initiated  a  research  and  development  program  to  provide  the 
means  for  the  development  of  embedded  training  as  a  part  of  the  systems 
acquisition  process.  One  of  the  approaches  in  the  program  is  the  development 
of  embedded  training  packages  in  exemplar  systems  selected  as  representative 
of  types  of  systems,  development  stage  and  approach,  and  ET  supportable  system 
capabilities . 

071  Hannaman,  D.L.,  Druker,  E.H.,  &  Childs,  E.L.  (1985).  Development  of  the 
functional  cequicemexvta-foc  aimulatioa  In  combined  aaa  training  (SIMCAX). 
(ARI  Research  Report  1394).  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences. 

Describes  SIMCAT,  a  computer  supported  battle  simulation  that  will  be  used  for 
conducting  research  on  how  to  train  command,  control,  and  communication  skills 
in  a  classroom  environment.  SIMCAT  will  consist  of  six  networked  micro¬ 
computers  (four  trainee  stations,  and  stations  for  the  opposing  force  and  the 
controller/  trainer) .  The  system  will  use  video  disks  and  computer  generated 
graphics  to  provide  audiovisual  cues,  inherent  in  tactical  situations,  to  all 
simulation  participants.  Voice  synthesis  and  speech  recognition  technologies 
will  permit  trainees  to  control  movement  and  firing  using  normal  communication 
protocols . 

068  Haiff,  H.M.,  Hollan,  J.D.,  &  Hutchins,  E.L.  (1986).  Cognitive  science 
and  military  training.  American  Psychologist.  41(10),  1131-1139. 

Discusses  how  the  massive  scale  and  diversity  of  training  needs  in  the 
military  render  it  an  enterprise  that  offers  extensive  opportunity  for  the 
application  of  cognitive  science.  Training  in  maintenance,  in  tactics,  and  in 
piloting  or  control  of  aircraft  present  several  issues  of  interest  to 
cognitive  scientists.  Four  recently  developed  military  training  systems 
illustrate  the  potential  for  cognitive  science  to  improve  military  training. 
These  systems  include  a  family  of  memorization  games  based  on  semantic 
networks;  a  simulator  for  steam  propulsion  plants  with  a  graphic,  schematic 
student  interface,  a  system  for  training  in  problems  of  relative  motion  that 
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provides  explicit  representations  of  spatial  concepts  and  problem-solving 
procedures;  and  a  method  of  building  a  new  cognitive  skill  for  air-intercept 
control  based  on  principles  for  the  development  of  automaticity.  These 
systems  illustrate  the  importance  of  making  relevant  knowledge  concrete  and 
explicit,  of  using  problem-solving  contexts  for  instruction  in  basic 
principles,  and  of  careful  management  of  information  processing  during 
learning. 
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Other  -  Additional  Titles 


Implementing  Embedded  Training  (ET) :  Volume  3  of  10:  The 

Role  of  ET  in  the  Training  System  Concept .  9  (141) 

A  Design  Architecture  for  an  Integrated  Training  System  Decision 

Support  System . 9  (082) 

The  development  and  evaluation  of  a  moving  target  engagement 

training  program  with  the  M16A1 .  27  (087) 

A  trilevel  interaction  design  model  for  pilot  part-task  training....  27  (137) 

Development  of  small  combat  arms  unit  leader  tactical  training 

techniques  and  a  model  training  system .  31  (152) 

Training  individuals  in  Army  units:  Comparative  effectiveness 
of  selected  training  extension  course  lessons  and  conventional 
methods .  47  (104) 

Conserving  instructional  training  resources  through  use  of  the 

personalized  system  of  instruction .  48  (177) 

Smart  technology  for  training:  Promise  and  current  status .  64  (065) 


FACTORS  INFLUENCING  TRAINING  OUTCOMES 

178williges,  B.H.,  williges,  R.C.,  &  Savage,  R.E.  (1979).  Predicting 
optimal  training  group  assignment.  Proceeding  of  the  Human  Factors 

Society  -  23rd  Annual  Meeting- 

Multiple  regression  equations  were  used  to  assign  40  students  to  fixed- 
difficulty  or  adaptive  training  based  upon  the  shorter  predicted  time-to-train 
score.  In  addition,  40  students  were  randomly  assigned  to  the  two  training 
conditions,  and  40  students  were  mismatched  to  training  based  upon  the  longer 
predicted  time  to  train.  Using  predicted  scores  to  match  students  to  training 
alternatives  resulted  in  a  47%  savings  in  training  time  over  random  assignment 
and  a  53%  savings  over  mismatched  assignment.  The  assignment  effect  was 
reliable  at  the  0.0001  level.  Future  research  will  examine  different 
categories  of  predictors,  additional  training  alternatives,  and  more  complex 
training  tasks. 

113  Maitland,  A.J.  (1980).  Using  predlot-nr  equations  to  assess  potential 

performance  of  armor  enlistees  (ARI  Research  Report  1240) .  Alexandria,  VA: 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Examines  research  findings  on  predicting  the  performance  of  U.S.  Army  armor 
crewmen  and  shows  how  these  findings  may  be  applied  in  the  current  training 
program.  Results  of  various  methods  of  assigning  groups  of  enlistees  to 
training  programs  illustrate  the  effects  of  different  decision  rules  on  the 
end-product  of  training.  With  the  method  of  prediction  supplied  in  this 
review,  the  performance  potential  of  groups  of  enlistees  may  be  assessed  from 
readily  available  test  scores,  and  the  result  of  assigning  these  individuals 
to  different  training  programs  may  be  estimated. 

025  Cohen,  A.D.  (1982) .  The  Hughes  design  analysis  system  and  instructor 
workload  in  operational  trainers.  Proceedings  of  the  Human  Faoi-m-« 

Society  -  26th  Annual  Meeting.  364-368. 

Computerized  simulator  trainers  require  varying  degrees  of  instructor 
participation  in  training  exercises  and  student  performance  evaluation.  An 
estimation  of  instructor  workload  before  the  system  has  been  fully  developed 
is  essential  for  an  effective  functional  allocation  of  human  and  computer 
capabilities.  One  approach  for  estimating  operator  workload  is  simulation. 

The  Hughes  Design  Analysis  System  (DAS)  is  an  interactive  and  graphic 
simulation  package.  The  present  paper  describes:  (1)  how  DAS  has  been  used 
for  an  instructor  work  load  analysis,  (2)  the  analysis  method,  (3)  the 
results,  and  (4)  the  recommendations  given  to  system  designers. 

105Knerr,  C.M.  (1984).  Armor  procedural  skills:  Learning  and  retention 

(ARI  Technical  Report  621).  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences. 

Investigated  the  learning  and  retention  of  a  subset  of  8  armor  tasks  selected 
to  represent  tasks  that  vary  in  length,  complexity,  and  extent  of  practice  in 
operational  units.  Two  data  collections  were  conducted.  One  collected 
performance  data  from  116  soldiers  in  operational  units;  the  other  utilized 
471  soldiers  attending  One  Station  Unit  Training  (OSUT)  in  Armor  Military 
Occupational  Specialty.  Ss  in  the  operational  unit  sample  had  been  out  of 
OSUT  entry  training  for  up  to  72  months.  Ss  in  the  OSUT  sample  participated 
in  a  series  of  task  learning  trials  for  two  tasks  followed  by  a  retention 
trial  approximately  4  weeks  later.  The  operational  unit  Ss  took  a  one-time 


73 


performance  test  on  all  8  tasks .  Results  are  consistent  with  previous  skill 
retention  research.  Multiple  regression  analysis  was  used  to  predict  the 
slopes  of  the  retention  function  for  each  task  for  the  combined  sample.  The 
prediction  equation  accounted  for  a  large  proportion  of  the  variance  when 
number  of  steps  in  the  task,  daily  practice  rate,  and  measures  of  complexity 
and  interference  were  used  as  predictors  of  skill  decay  rate.  Results  of  the 
OSOT,  unit,  and  combined  samples  support  a  representation  of  the  skill 
retention  curve  in  which  rapid  decay  occurs  soon  after  training  with  little 
change  in  performance  for  samples  tested  later. 

040  Dunbar,  S.B.,  Mayekawa,  S.,  &  Novick,  M.R.  (1986).  Simultaneous 

estimation  of  regression  functions  for  Marine  Corps  technical  training 
specialties.  Journal  of  Educational  Statistics.  11(4),  275-292. 

Considers  the  application  of  Bayesian  techniques  for  simultaneous  estimation 
to  the  specification  of  regression  weights  for  selection  tests  used  in  various 
technical  training  courses  in  the  Marine  Corps.  Results  of  a  method  for  in¬ 
group  regression  suggest  that  a  hypothesis  of  conplete  generalization  of  the 
predictor-criterion  relationships  across  training  courses  in  a  given  category 
would  only  be  retained  for  a  carefully  selected  subset  of  courses  and  not  for 
all  groups  included  in  the  analysis. 

067  Hagman,  J.D.,  &  Hayes,  J.F.  (1986).  Cooperative  learning :  Effects  of 
task,  reward,  and  group  size  on  individual  achievement  (ARI  Technical 
Report  704).  Alexandria,  VA:  U.S.  Arm/  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

Investigated  whether  cooperative  learning  can  effectively  promote  individual 
achievement,  using  360  military  trainees.  Exp  I  compared  the  performance  of 
280  trainees  after  they  had  completed  practical  exercises  under  cooperative  or 
individual  learning.  Results  reveal  that  cooperative  learning  improved 
individual  test  scores  when  coupled  with  a  group  reward  contingency,  and 
significant  benefits  occurred  once  group  size  reached  four  members.  Exp  II 
employed  80  trainees  to  determine  why  group  reward  was  necessary  for  obtaining 
enhanced  individual  achievement  under  cooperative  learning.  Two  potential 
hypotheses  were  tested:  (1)  Groip  reward  effects  are  caused  by  increased 
individual  trainee  motivation  to  learn  resulting  from  group  pressure  to 
perform;  and  (2)  group  reward  encourages  group  mates  to  share  information,  and 
this  peer  tutoring  facilitates  individual  learning.  Results  support  the  peer 
tutoring  hypothesis. 

016  Brooks,  J.E.  (1987).  An  Instructor's  guide  for  iinplmnonKno  cooperative 
learning  in  the  equipment  xecoxds  and  parts  specialist  course  (AD  A191 
660).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

Cooperative  learning  is  a  new  approach  that  is  being  implemented  at  the  U.S. 
Army  Quartermaster  School  for  training  Equipment  Records  and  Parts  Specialist 
(MOS  76C) .  This  report  describes  the  background,  development,  and  application 
of  an  Instructor's  Guide  that  was  developed  by  the  Army  Research  Institute 
(ARI)  to  familiarize  course  instructors  with  the  specific  procedures  used  in 
implementing  the  techniques.  The  guide  itself  is  deluded  in  an  appendix. 

The  guide  is  divided  into  sections  that  are  appr  te  to  the  particular 

information  needs  of  the  three  different  instruc  earns  that  teach  the  76C 

course.  It  also  contains  general  background  information  about  the  cooperative 
learning  technique  and  the  reasons  for  its  implementation,  although  the  main 
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purpose  of  the  guide  is  to  provide  instructors  with  essential  information,  it 
may  also  be  used  as  a  model  for  developing  similar  guides  for  use  in  other 
courses  that  adopt  the  cooperative  learning  approach. 

017  Brooks,  J.E.,  Cormier,  S.M.,  Oressel,  J.D.,  Glaser,  M.,  Knerr,  B.W., 

Thorenson,  R.  (1987) .  Cooperative  learning:  A  new  approach  for  training 
equipment  records  and  parts  specialists  (ARI  Technical  Report  760) . 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

This  research  investigated  the  usefulness  of  cooperative  learning  for 
promoting  individual  achievement  in  the  Equipment  Records  and  Parts  Specialist 
Course  (MOS  76C) .  cooperative  learning  students  in  four-member  groups  with 
group  reward  were  compared  to  individual  learning  students  on  measures  of 
course  achievement,  task  completion  speed,  and  study  hall  attendance.  Student 
and  instructor  attitudes  towards  cooperative  learning  were  also  assessed.  A 
trial  implementation  phase  resulted  in  several  procedural  modifications, 
whereas  the  second  phase  was  a  controlled  evaluation.  The  results  showed  that 
cooperative  learning  reduced  academic  recycling  by  about  one  half  and  had  no 
effect  on  the  test  performance  of  most  students.  Students  working  in  groups 
on  practical  exercises  (PEs)  made  fewer  errors  than  students  working  alone. 
Groups,  however,  often  took  longer  to  complete  PE  assignments.  Normal 
training  schedules  were  not  disrupted  by  increased  PE  completion  times  or  by 
increases  in  study  hall  attendance  resulting  from  group  rewards.  Most 
students  and  instructors  liked  cooperative  learning.  It  was  concluded  that 
cooperative  learning  is  a  feasible,  low-cost  approach  that  offers  cost  savings 
through  reduced  student  recycling.  Procedural  modifications  that  may  increase 
achievement  benefits  are  noted,  and  future  research  directions  are  suggested. 

083  Hogan,  J.,  Arneson,  S.,  &  Salas,  E.  (1987).  Individual  differences  in 

military  training  environments: _ Four  areas  of  research  (ntsc  Tech  Report 

87-003) .  Orlando,  FL:  U.S.  Naval  Training  Systems  Center. 

Addresses  the  variety  of  individual  learning  and  ability  differences  that  are 
brought  to  the  training  environment  in  terms  of  trainees  cognitive  strategies, 
trainees'  noncognitive  characteristics,  and  aptitude-performance  interactions. 
Summarized  findings  are  used  to  develop  a  decision  model  for  a  training 
program  design  that  involves  classification  of  training  tasks,  selection  of 
personnel  for  training,  and  specification  of  instructional  strategies  for 
optimal  learning.  Task  types  include  the  realistic,  investigative,  artistic, 
social,  enterprising,  and  conventional.  Personality  factors  include 
"intelligence",  adjustment,  prudence,  ambition,  sociability,  and  likeability. 

093  Jones,  M.B.,  Kennedy,  R.S.,  Kuntz,  L.A.,  &  Baltzley,  D.R.  (1987). 

Isoperformance:  Trading  off  selection,  training,  and  equipment  variations 

to  maintain  the  same  level  of  systems  performance.  Proceedings  Qf . the 

Human  Factors  Society  -  31st  Annual  Meeting.  634-637. 

121  Mumford,  M.D.,  Harding,  F.D.,  Fleishman,  E.A.,  &  Weeks,  J.L.  (1987).  An 
empirical  system  for  assessing  the  impact  of  aptitude  requirement 
adjustments  on  Air  Force  initial  skills  training  (U.S.  AFHRL  Technical 
Report  86-19).  San  Antonio,  TX:  U.S.  Air  Force  Human  Resources 
Laboratory. 

Developed  a  decision  support  system  for  Air  Force  personnel  and  training 
management.  Measures  of  student  input,  course  content,  and  training  outcome 
were  obtained  for  39  initial  skills  courses  and  5,000  trainees.  A 
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hypothetical  model  of  initial  skills  training  is  presented.  When  cross- 
validated  using  9  additional  training  courses  and  1,000  trainees,  the  model 
yielded  predicted  training  outcomes  that  were  consistent  with  actual  training 
outcomes . 

122  Mumford,  M.D . ,  Weeks,  J.L.,  Harding,  F.D.,  &  Fleishman,  E.A.  (1988) . 
Relations  between  student  characteristics,  course  content,  and  training 
outcomes:  An  integrative  modeling  effort.  Journal  of  Applied  Psychology. 
21(3),  443-456. 

We  have  attempted  to  formulate  a  general  multivariate  model  describing  the 
relationships  among  individual  attributes,  course-content  variables,  and 
performance  indexes  in  Air  Force  training  programs.  A  series  of  interviews 
with  course  instructors  and  training  managers  led  to  the  identification  of  six 
student-characteristics,  16  course-content,  and  seven  training-performance 
variables  held  to  be  important  descriptors  of  the  training  process,  regardless 
of  the  particular  program  under  consideration.  Measures  of  these  variables 
were  then  obtained  in  39  training  courses  containing  5,078  students.  The 
relationships  observed  among  these  variables  were  used  to  develop  a 
hypothetical  model  that  was  submitted  to  formal  structural  analysis.  This 
structural  model  was  cross-validated  on  a  sample  of  9  additional  courses 
containing  890  students.  In  cross-validation,  the  model  predicted  training 
performance  criteria  with  sufficient  accuracy  to  be  considered  a  valid 
descriptor  of  the  technical  training  process. 

051  Fleishman,  E.A.,  &  Mumford,  M.D.  (1989).  Individual  attributes  and 

training  performance.  In  I.L.  Goldstein  and  Associates  (Ed.),  Training  and 
ftoralnpmanr  in  organizations  (pp.  183-255) .  San  Francisco,  CA:  Jossey- 
Bass  Publishers. 

Reviews  literature  on  individual  differences  as  expressed  in  abilities, 
skills,  and  knowledges  and  the  impact  of  these  differences  among  people  on 
training  outcomes.  Stresses  the  importance  of  learners'  characteristics  on 
learning  and  training  of  such  variables  as  aptitudes,  reading  level,  academic 
motivation,  educational  level,  educational  preparation,  and  age.  Discusses  at 
length  the  Ability  Requirements  Taxonomy  system  with  50  ability  categories. 
Describes  how  to  link  abilities  and  job/task  requirements  in  training,  skill 
acquisition,  training  system  design,  and  training  research. 
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Factors  Influencing  Training  Outcomes  -  Additional  Titles 

Job  performance  aiding  systems  technology  selection  algorithm .  13  (Oil) 

Computers,  learning  styles,  and  instructional  materials — are 

they  related? .  16  (130) 

Developing  the  potential  of  low  ability  personnel .  23  (089) 

Accommodating  instruction  to  student  characteristics:  Trends 

and  issues .  23  (044) 

Chaparral  skill  retention .  24  (150) 

Retention  of  military  tasks:  A  review .  26  (066) 

Part-task  training  for  tracking  and  manual  control .  27  (180) 

Using  predictor  equations  to  assess  potential  performance .  38  (111) 

Technical  training:  Development  of  instruction  treatment 

alternatives .  48  (161) 

Transfer  of  training:  A  review  and  directions  for  future  research..  49  (006) 

The  instructional  effectiveness  of  three  levels  of  graphics 

displays  for  computer-assisted  instruction .  63  (119) 

An  examination  of  the  research  evidence  for  computer-based 

instruction  in  military  training .  65  (146) 


COSTING  AND  RESOURCING 


Coat  and  Raaourca  Estimation 

043  Eggemeier,  F.T.  i  Klein,  G.A.  (1978).  Life  cycle  costing  of  simulated 
vs.  actual  equipment  for  intermediate  maintenance  training.  Proceedings 
nf  the  Human  Factors  Society  -  22nd  Annual  Meeting. 

Life  cycle  cost  estimates  of  training  equipment  for  F-16  Avionics  Intermediate 
Station  personnel  were  developed.  The  major  purpose  was  to  compare  the  cost 
of  intermediate  level  maintenance  training  when  conducted  on  simulated  versus 
actual  avionics  test  equipment.  This  was  the  initial  phase  of  a  planned  two- 
part  effort.  The  analysis  was  therefore  limited  to  estimates  of  training 
device  acquisition  and  maintenance  costs.  Total  estimated  fifteen  year  costs 
for  simulated  equipment  trainers  were  approximately  50%  less  than  comparable 
estimates  for  actual  equipment  trainers. 

117  McCullough,  J.M.  (1984).  To  measure  a  vacuum.  Training  and  Development 
Journal.  1&(6),  68-70. 

Describes  the  Deficiency  Analysis  Review  Technique  (DART) ,  a  method  for 
justifying  training  needs  and  quantifying  results  of  training.  DART  may  cause 
managers  to  become  more  receptive  to  training  because  it  quantifies  the  cost 
of  not  doing  the  training.  The  system  involves  systematically  identifying  and 
measuring  deficient  operations  and  unnecessary  expenses,  which  then  become  the 
target  of  formal  employee  training.  The  steps  in  DART  include  problem 
identification,  goal  development,  methodology  development,  cost/benefit 
analysis,  documentation,  course  development,  and  course  implementation. 
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Costing  and  Resourcing 

Cost  and  Resource  Estimation  -  Additional  Titles 

Maintenance  training  and  simulation:  Design  processes  and 

evaluation  criteria .  7  (015) 

A  Design  Architecture  for  an  Integrated  Training  System  Decision 

Support  System .  9  (082) 

A  technique  for  choosing  cost  effective  instructional  delivery 

systems .  11  (012) 

Training  device  operational  readiness  assessment  capability 

(DORAC)  :  feasibility  and  utility .  16  (077) 

A  review  of  models  of  cost  and  training  effectiveness  analysis  . 

(CTEA)  Volume  I:  Training  effectiveness  analysis .  19  (059) 

Designing  training  devices:  The  optimization  of  simulation- 

based  training  systems .  20  (156) 

Use  of  a  knowledge-based  system  to  assess  aircrew  training 

requirements  as  part  of  conceptual  design .  20  (158) 

First  may  or  may  not  be  best,  or  cost  and  training  effectiveness 

analysis  in  comparing  training  approaches  .  37  (124) 

A  constrained  maximization  extension  to  incremental  transfer 

effectiveness,  or,  how  to  mix  your  training  technologies .  38  (022) 

Cost  and  training  effectiveness  analysis  in  the  Army  Life  Cycle 

Systems  Management  Model .  38  (115) 

A  forecasting  method  for  training  effectiveness  analysis .  39  (032) 

Comparison  of  training  transfer  and  effectiveness  models .  40  (102) 

A  review  of  models  of  cost  and  training  effectiveness  analysis  . 

(CTEA)  Volume  II:  Cost  analysis .  43  (002) 

Research  and  development  into  a  comprehensive  media  selection  model  43  (018) 

Training  effectiveness  and  cost  iterative  technique  (TECIT) . 

Volume  II:  Cost  effectiveness  analysis .  44  (003) 

Cost  and  information  effectiveness  analysis:  An  improved 

methodology  (Volume  II) .  55  (014) 

Cost  and  information  effectiveness  analysis:  An  improved 

methodology  (Volume  I) .  55  (079) 

Actual  vs.  simulated  equipment  for  aircraft  maintenance  training: 

Cost  implications  of  the  incremental  vs.  the  unique  device .  61  (173) 
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factor*  Affecting  Cost 


026  Cohen,  E.  (1973) .  Training  time  as  a  selection  criterion  for  entrants  to 
individualized  training  programs.  Psychological  Reports.  .22.(3,  Pt  1), 
715-718. 

Considers  that  the  criterion  for  selection  of  entrants  to  training  and 
educational  programs  is  almost  always  success  in  the  training  program  itself; 
occasionally  the  criterion  is  performance  on  the  job  for  which  the  training 
program  is  prerequisite.  The  advent  of  individualized  training  programs,  in 
which  each  trainee  proceeds  at  a  rate  determined  by  his  own  learning  speed 
rather  than  in  "lock  step"  with  the  rest  of  his  class,  opens  the  possibility 
of  selecting  the  faster  learners  and  thus  effecting  substantial  economies. 
Some  of  the  implications  of  the  "select  for  speed  of  learning"  approach  are 
discussed. 
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Factors  Affecting  Cost  -  Additional  Titles 


Use  of  a  knowledge-based  system  to  assess  aircrew  training 

requirements  as  part  of  conceptual  design .  20  (158) 

Training  approaches  and  data  entry  methods  for  semiautomatic 

command  and  control  systems .  24  (054) 

Increasing  technical  training  efficiency:  Peer  training  via 

computer-assisted  instruction .  25  (036) 

Conserving  instructional  training  resources  through  use  of  the 

personalized  system  of  instruction .  48  (177) 

Actual  versus  simulated  equipment  for  aircraft  maintenance 
training:  Cost  implication  of  the  incremental  versus  the 

unique  device .  61  (173) 

Factors  influencing  the  logistical  support  of  maintenance 

training  simulators:  The  development  of  a  handbook .  61  (021) 

Predicting  optimal  training  group  assignment .  73  (178) 

Life  cycle  costing  of  simulated  vs.  actual  equipment  for 

intermediate  maintenance  training .  79  (043) 

To  measure  a  vacuum .  79  (117) 
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MISCELLANEOUS 


058  Goclowski,  J.C.,  King,  G.F.,  Ronco,  P.G.,  S  Askren,  W.B.  (1978). 

Tntagrafc ion  and  application  of  human  resource  technologies  in  weapon 
system  design:  Consolidated  data  base  functional  specif ication  (U.S. 

AFHRL  Technical  Report,  No.  78-6(3)).  San  Antonio,  TX:  U.S.  Air  Force 
Human  Resources  Laboratory. 

Describes  the  consolidated  data  base  (CDB)  required  to  support  the  application 
of  the  coordinated  human  resource  technology  (CHRT)  on  a  weapon  system 
acquisition  program.  The  major  categories  of  data  stored  in  the  CDB  relate  to 
reliability,  maintainability,  maintenance  manpower,  operations  manpower, 
training,  and  job  guides  for  both  maintenance  and  operations,  and  system 
ownership  costs.  The  CDB  may  also  be  used  for  operational  and  support 
planning  after  deployment.  As  developed  for  application,  the  CDB  is  unique  to 
each  weapon  system.  It  expands  in  detail  with  time  as  the  weapon  system 
acquisition  cycle  progresses.  The  CDB  is  dynamic  in  nature  representing 
alternatives  being  considered  as  well  as  baseline  approaches.  It  has, 
therefore,  been  designed  for  frequent  update  and  expansion. 

061  Goldstein,  I.L.  (1978) .  A  systems  approach  to  studying  organizations: 
selection,  training,  performance.  Proceedings  of  the  Human  Factors 
Society  -  22nd  Annual  Meeting. 

This  presentation  explores  the  idea  that  training  analysts  have  not  fully 
utilized  the  rich  source  of  data  which  can  stem  from  the  examination  of 
training  programs.  Several  approaches  including  individual  difference 
methodology,  trainee  expectations  and  training  process  measures  are  discussed. 

073  Hanson,  V.L.,  &  Purifoy,  G.R.  (1979).  TSM  guide  to  training  development 
and  acquisition  for  major  systems.  Catalog  of  Selected  Documents  in 

Psychology,  A,  99-100. 

170  Swezy,  R.W,  (1979) .  An  application  of  a  multi-attribute  utilities  model 
to  training  analysis.  Human  Factors.  21(2),  183-189. 

The  multi-attribute  utilities  model  of  W.  Edwards  et  al.  (1975)  Bayesian- 
oriented  decision-making  paradigm,  was  applied  to  a  decision-making  problem  in 
a  military  training  analysis  situation.  The  model  is  compared  to  a  second 
simple  judgment  analysis  model  using  the  same  input  data. 

053  Freda,  J.S.  (1980).  Army  training  technology  transfer:  A  systems  model 
(ARI  Research  Report  1241) .  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences. 

Innovations  in  training  technology  must  be  transferred  from  the  researcher  to 
the  user  to  be  effective.  This  system  concept  paper  presents  a  framework  for 
the  establishment  of  an  Army  training  technology  transfer  program,  as  well  as 
suggestions  to  improve  the  acceptance  and  use  of  training  research  products. 

Training  technology  transfer  is  defined  as  the  process  by  which  research 
results  become  implements  of  useful  change  in  operational  training.  The  user 
can  be  civilian  or  military,  functionally  involved  at  levels  from  high-level 
review  to  the  trainer  in  the  field.  The  systems  model  developed  here  uses  a 
linear  approach  to  describe  the  four  steps  of  technology  transfer  in  the  Army: 
analysis  of  requirements;  research,  development,  test,  and  evaluation  of 
solutions;  dissemination  of  findings;  and  institutionalization.  Within  these 


steps,  specific  issues  include  assessment  of  military  needs,  consideration  of 
the  appropriateness  of  current  RDT&E  funding,  user  acceptance  of  new  products, 
and  the  transition  from  innovation  to  policy,  as  well  as  prediction 
methodology  and  recommendations  for  ongoing  monitoring,  evaluation,  and 
feedback.  Recurring  throughout  the  transfer  process  are  issues  of  sponsorship 
and  the  self-renewal  capability  of  the  research  product. 

A  bibliography  on  training  transfer  is  organized  in  terms  of  specific 
activities  within  the  model. 

062  Goldstein,  I.L.  (1980).  Training  in  work  organizations.  Annual  Review 
of  Psychology.  21,  229-272. 

Examines  issues  concerning  systematic  modes  of  instruction  designed  to  produce 
environments  that  shape  behavior  to  satisfy  stated  objectives.  The  literature 
and  state  of  the  field  are  reviewed  in  the  areas  of  (a)  general  systems 
philosophy  (needs  assessment,  the  instructional  process,  and  evaluation) ;  (b) 
approaches  to  training  issues  (developmental  psychology  aspects;  training  for 
hard-core  unemployed,  women,  and  the  aged;  individual  differences;  and  fair 
employment  practices);  (c)  training  for  various  functions  (police, 
management) ;  and  (d)  training  methods  (sensitivity  training,  simulations,  and 
computer-assisted  and  programmed  instruction) . 

094  Jones,  M.B.  (1981)  .  Converoence-diveraence  with  extended  practice: 

Three  applications  (NBDL-81R010) .  New  Orleans,  LA:  U.S.  Naval 
Biodynamics  Laboratory. 

The  author  and  colleagues  (1980)  introduced  the  concept  of  "convergence/ 
divergence  with  practice"  to  indicate  changes  in  the  correlations  between  a 
task  and  an  external  measure  over  time.  The  present  paper  develops  this 
concept  in  3  settings:  differential  retention  over  long  periods  of  no 
practice,  personnel  selection  and  classification,  and  the  identification  of 
latent  factors.  Military  research  projects  using  video  games  or  code 
substitution  tasks  are  reviewed  to  demonstrate  that  the  correlation  between  a 
S's  terminal  level  of  performance  in  practice  and  retraining  should  increase 
with  the  number  of  retraining  trials.  Implications  are  discussed  for 
personnel  selection  in  cases  in  which  the  criterion  develops  in  a  series  of 
stages  or  phases  and/or  factor-an  vtic  attempts  to  identify  abilities. 

095  Jones,  M.B. ,  Kennedy,  R.S.,  &  Bittner,  A.C.  (1981).  Video  games  and 
convergence  or  divergence  with  practice  (NBDL-81R010,  1-5) .  New  Orleans, 
LA:  U.S.  Naval  Biodynamics  Laboratory. 

Discusses  the  use  of  video  games  as  psychological  tests  of  skill  acquisition 
and  learning  and  their  fulfillment  of  the  requirements  of  stabilization  and 
task  definition  in  performance  testing.  Cross-correlational  data  from  13  U.S. 
Navy  enlisted  personnel  on  air  combat  maneuvering  simulation  and  a  commercial 
video  game  reveal  that  performance  skills  on  these  2  measures  systematically 
increased  over  15-day  trials.  Application  of  video  games  to  pilot  selection 
and  training  is  discussed. 
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133  Rigg,  K.E.,  Gray,  B.B.,  &  Tillman,  B.W.  (1982).  Estimating  skill 

retention  from  initial  training.  Proceedings  of  the  Human  Factors  Society 
-  26th  Annual  Meeting.  263-265. 

Performance  of  a  procedural  skill  is  known  to  decay  in  the  absence  of 
practice;  for  many  tasks  significant  decay  is  observed  over  periods  as  short 
as  a  few  weeks .  This  decay  in  performance  has  been  observed  in  such  diverse 
groups  as  astronauts,  chemical  operators  and  Army  riflemen.  Unsurprisingly, 
many  Army  basic  tasks  are  forgotten  in  the  period  between  basic  training  and 
unit  assignment.  This  report  presents  the  findings  of  a  research  effort  to 
apply  a  mathematical  training  model  developed  by  McFann-Gray  &  Associates, 

Inc.  (MGA) ,  to  Army  basic  training:  (1)  to  describe  initial  training;  (2)  to 
estimate  end-of-course  performance;  and  (3)  to  estimate  the  level  of  field 
performance  after  a  retention  interval  without  practice. 

165  Sticht,  T.G.  (1982) .  Basic  skills  in  defense  (HumRRO  Professional  Paper, 
PP  3-82) .  Alexandria,  VA:  Human  Resources  Research  Organization. 

A  literature  review  indicates  that  a  debate  about  basic  skills  training  for 
recruited  military  personnel  has  been  unresolved  for  200  years.  Those  against 
basic  skills  training  cite  costs,  ineffectiveness,  and  undesirable 
performance.  Those  for  basic  skills  training  cite  demographics,  wartime 
needs,  and  equality  of  performance  among  highly  literate  and  less  literate 
personnel.  Selection,  classification,  training,  and  job  performance  studies 
have  yielded  results  that  support  both  sides.  Congress  has  directed  that  high 
school  diploma  programs  be  available  during  off-duty  hours  and  all  services 
require  some  education  credentials  for  noncommissioned  officer  rank. 

Experience  with  basic  skills  programs  in  the  various  services  has  been 
generally  favorable  with  job-oriented  basic  skills  training  demonstrating  the 
positive  effect  of  literacy  on  job  performance.  It  is  suggested  that  if  the 
current  trend  toward  job-  and  career-oriented  basic  skills  education 
continues,  the  debate  will  be  resolved. 

050  Fisk,  A.D.,  Scerbo,  M.W.,  &  Schneider,  W.  (1983).  Issues  in  training  for 
Skilled  performance.  Proceedings  of  the  Human  Factors  Society  -  27th 
Annual  Meeting,  392-396. 

Four  fallacies  concerning  training  for  skilled  performance  that  are  often 
implicitly  assumed  in  training  programs  are  discussed.  These  fallacies  are: 

(1)  Practice  makes  perfect;  (2)  Training  of  the  total  skill  is  optimum;  (3) 
the  goal  of  training  is  to  produce  accurate  performance;  and  (4)  skill 
training  is  intrinsically  enjoyable.  Automatic/ controlled  processing  theory, 
which  emphasizes  how  training  may  be  done  to  avoid  these  fallacies,  is  briefly 
discussed.  Finally,  11  training  guidelines  are  provided  for  the  optimization 
of  skill  training. 

074  Harmon,  J.  (1984) .  Three  years  of  evaluation  of  the  Army's  basic  skills 
education  program  (ARI  Research  Report  1380).  Alexandria,  VA:  U.S.  Army 
Institute  for  the  Behavioral  and  Social  Sciences. 

Summarizes  3  years  of  evaluation  research  on  the  U.S.  Army's  Basic  Skills 
Education  Program,  which  provides  enabling  skills  in  literacy,  language, 
arithmetic,  computation,  and  speaking.  Analyses  of  standard,  pilot,  and 
revised  programs  as  well  as  programs  under  development  are  described.  Overall 
findings  reveal  that  all  programs  tend  to  move  participants  in  the  direction 
of  Army  goals,  although  a  substantial  number  of  soldiers  fail  to  meet  program 
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criteria.  Factors  that  may  dilute  program  effectiveness  include  teachers' 
lack  of  specialized  training,  the  wide  range  of  skill  levels  within  classes 
and  personnel  turbulence. 


136  Risser,  D.T.,  &  Berger,  P.  (1984) .  Army  HARDMAN:  Its  origin, 

evaluation,  implementation  to  date.  Proceedings  of  the  Human  Factors 
Society  -  28th  Annual  Meeting.  716-720. 

The  HARDMAN  Comparability  Methodology  was  developed  by  the  Navy  to  estimate 
human  resource  requirements  during  the  early  phases  of  weapon  system 
development.  The  Army  has  modified  the  system  to  support  its  own  needs  in 
this  area.  Four  real  system  applications  in  the  Army  and  an  evaluation  by  the 
Jet  Propulsion  Laboratory  indicated  that  it  is  useful,  usable,  and  reasonably 
credible,  and  provides  a  near-term  solution  to  the  Army's  needs.  An  Army  Guide 
is  under  development . 

181  Zimmerman,  W.,  Butler,  R.,  Gray,  V.,  &  Rosenberg,  L.  (1984).  Evaluation 
of  the  HARDMAN  (hardware  vs.  manpower)  comparability  methodology  (ARI 
Technical  Report  646).  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

Evaluated  the  U.S.  Army's  HARDMAN  methodology  for  projecting  manpower, 
personnel,  and  training  (MPT)  requirements  to  support  new  acquisitions.  Tests 
included  a  general  examination  of  the  basic  reasonableness  of  the  technique;  a 
reliability  measure  of  how  well  other  trained  individuals,  given  the  same  data 
inputs,  can  derive  the  same  answers  as  an  already-completed  test  case;  a 
qualitative  accuracy  check  based  on  conversations  with  individuals  who  have 
already  applied  the  same,  or  a  similar,  methodology;  and  a  measure  of  how 
accurately  the  methodology  must  predict  the  real  world.  The  evaluation 
reveals  that  the  methodology  conforms  fairly  well  with  both  the  Army's  MPT 
users'  needs  and  other  accepted  manpower  modeling  techniques.  Audits  of  3 
completed  HARDMAN  applications  and  the  reliability  study  reveal  a  need  to 
consolidate  select  judgmental  portions  of  the  methodology  and  to  improve  those 
aspects  of  the  technique  used  for  making  MPT  support  and  budget  decisions. 
Results  suggest  that  the  manpower  life-cycle  cost  component  is  only  marginally 
sensitive  to  changes  in  other  related  cost  variables. 

147  Schneider,  W.  (1985) .  Training  high-performance  skills:  Fallacies  and 
guidelines.  Human  Factors,.  22(3),  285-300. 

Argues  that  training  programs  for  developing  high-performance  skills  (HPSs) 
are  often  based  on  assumptions  that  may  be  appropriate  for  simple  skills. 

These  assumptions  can  be  fallacious  when  extended  to  HPSs.  Six  fallacies  of 
training  are  described,  and  empirical  characteristics  of  HPS  acquisition 
(e.g.,  long  acquisition  periods,  heterogeneity  of  component  learning, 
developing  of  inappropriate  strategies)  are  reviewed.  A  tentative  set  of 
guidelines  for  the  acquisition  of  HPSs  is  described. 

019  Carnevale,  A.P.  (1986).  The  learning  enterprise.  Training  and 
Development  Journal.  12(1),  18-26. 

Discusses  the  size,  scope,  and  influence  of  present-day  training  and 
development.  It  is  suggested  that  most  American  employees  learn  what  they 
need  to  know  to  do  their  jobs  in  schools  and  on  the  job,  formally  and 
informally.  School  tends  to  be  most  important  for  managers,  professionals, 
technical  and  sales  employees,  and  people  in  publicly  regulated  jobs.  Most 
formal  training  in  the  workplace  concentrates  in  the  25-44-year-old  age  group 


and  among  white-collar  managers,  professionals,  and  clericals.  Informal 
training  is  most  important  in  blue-collar  jobs .  Research  indicates  that  the 
split  between  behavioral  and  cognitive  training  and  development  shows  that 
roughly  two-thirds  of  all  formal  training  and  development  is  cognitive  and 
that  one-third  is  behavioral.  Workplace  training  inside  and  out,  costs  and 
benefits  of  workplace  learning,  and  the  growth  and  decline  in  the  training  and 
development  function  in  companies  are  discussed.  Tables  are  included  that 
represent  trends  in  training  and  development,  such  as  sources  of  training  and 
retraining  by  occupational  groups,  employee  training  by  industry,  changes  in 
employee  participation  in  training  and  use  of  outside  training  resources,  and 
demand  for  trainers. 

084  Holt,  V.E .  (1987) .  Issues  in  psychological  research  and  application  in 
transfer  of  training  (ARI  Research  Note  87-65).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 

This  research  note  is  a  collection  of  papers  and  summary  recommendations 
resulting  from  a  two-day  workshop  focused  on  training  transfer.  Supported  by 
the  Army  Research  Institute,  the  workshop  featured  presentations  by  academic, 
non-academic,  and  military  laboratory  scientists  on  psychological  research  and 
applications  related  to  transfer  of  training.  Among  the  specific  topics  dealt 
with  are:  the  development  of  cognitive  simulation  models,  skills  development 
methods,  the  need  for  intelligent  job  aids,  cognitive  task  analysis,  and 
methods  for  measuring  job  performance.  Recommendations  for  further  research 
and  applications  are  provided. 

153  Sicilia,  G.T.  (1988) .  Developing  a  training  data  architecture  that  ties 
it  all  together.  Proceedings  of  the  Human  Factors  Society  -  32nd  Annual 
Meeting,  1198-1201. 

Everyone  involved  in  quantifying  the  human  part  of  the  defense  equation  has 
long  been  frustrated  by  lack  of  consistent,  "reproducible'’  and  comprehensive 
data.  This  void  is  especially  evident  in  the  training  and  performance  data 
area.  The  Defense  Training  and  Performance  Data  Center  (TPDC)  was  established 
to  help  redress  this  problem. 

041  Duncan,  C.S.,  &  Greenston,  P.M.  (1988).  Occupation  data  base  planning 
and  design.  Proceedings  of  the  Human  Factors  Society  -  32nd  Annual 
Meeting.  1202-1206. 

The  Occupation  Data  Base  under  development  at  TPDC  consists  of  three  files. 

The  first  file  describes  military  occupations,  providing  information  about  the 
attributes  of  the  occupation.  It  addresses  the  question:  what  does  the 
occupation  look  like?  The  second  file  describes  occupations  from  a 
"demographic"  perspective,  or  more  simply  put,  answers  the  question:  who  are 
the  people  performing  the  job?  It  provides  a  statistical  profile  (current  and 
historical)  of  the  characteristics,  experience,  and  behavior  of  the  people 
serving  in  each  military  occupation.  The  third  file  provides  a  data  file  on 
training  programs,  resources,  methods  and  media  as  these  elements  relate  to 
military  occupations.  All  occupations  within  the  enlisted,  warrant  officer, 
and  officer  communities  are  covered  in  each  service,  both  active  and  reserve 
components.  These  three  files  are  being  built  from  current  and  historical 
data  and  will  be  updated  regularly,  thereby  providing  a  comparative 
perspective  with  which  senior  service  planners  can  make  informed  decisions  on 
defense  training  issues. 
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037  Driskell,  J.E.,  £  Olmstead,  B.  (1989).  Psychology  and  the  military: 

Research  applications  and  trends.  American  Psychologist,  11(1),  43-54. 

Perhaps  no  other  institution  has  been  as  inextricably  linked  with  the  growth 
and  development  of  psychology  as  the  military.  This  symbiotic  relationship, 
bora  of  the  expediency  of  World  War  I,  rests  on  two  roles:  (a)  the  military  as 
a  test-bed  or  applied  laboratory  for  the  examination  of  psychological 
processes,  and  (b)  the  military  as  an  impetus  to  initiate  and  direct  research 
and  innovation  in  psychology.  In  this  article,  we  examine  three  areas  of 
research  that  in  historical  perspective  have  most  prospered  from  this 
relationship:  selection  and  classification,  training,  and  human  factors.  We 
address  topics  that  hold  promise  for  future  collaboration  and  describe  how  the 
military  manages  research  and  development.  Finally,  to  assist  those 
interested  in  research  opportunities  with  the  military,  we  profile  the  primary 
behavioral  science  research  organization  within  the  Department  of  Defense. 

081  Herlihy,  D.,  Bondaruk,  J.,  Nicholas,  G.,  Guptill,  R.,  Park,  J.  (1990). 

Hardware  versus  manpower  compatibility  methodology. _ Volume  I.. _ Overview 

and  manager's  guide.  Wilmington,  MA:  Dynamics  Research  Corporation. 

The  Army  Hardware  vs.  Manpower  (HARDMAN)  Comparability  Methodology  (HCM)  is  a 
six-step  process  for  determining  a  weapon  system's  manpower,  personnel,  and 
training  (MPT)  requirements.  It  provides  a  structured  approach  for  early  MPT 
estimation  based  on  comparability  analysis,  an  analytic  system  that  uses 
knowledge  about  similar  existing  systems  and  technological  growth  trends  to 
project  the  MPT  requirements  of  proposed  new  systems.  The  HCM's  six 
interrelated  steps  are  systems  analysis,  manpower  requirements  analysis, 
personnel  pipeline  analysis,  training  resource  requirements  analysis,  inpact 
analysis,  and  trade-off  analysis.  The  HCM  has  been  successfully  applied  to  a 
range  of  weapons  systems,  including  air,  armor,  artillery,  infantry,  air 
defense,  command  and  control,  and  intelligence  systems.  The  product 
improvement  program  for  HCM  made  major  revisions  to  the  existing  HCM  guide. 

The  scope  has  been  expanded  to  include  several  new  areas;  existing  procedures 
have  been  revised,  refined,  and  clarified;  and  the  entire  guide  has  been 
rewritten  to  achieve  greater  charity,  consistency,  and  completeness.  This 
volume  addresses  the  planning  and  conducting  of  an  HCM  analysis.  Procedures 
are  provided  for  determining  the  analysis  scope  and  estimating  the  resources 
required  for  the  analysis.  Preparation  of  the  quality  assurance  plan  and 
establishment  of  the  consolidated  database  are  explained.  The  relationship 
between  HCM  results  and  various  Army  MPT  documents  is  also  discussed. 
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